Multilingualism and Language Teaching 9 Ruth Trüb An Empirical Study of EFL Writing at Primary School An Empirical Study of EFL Writing at Primary School Multilingualism and Language Teaching Herausgegeben von Thorsten Piske (Erlangen), Silke Jansen (Erlangen) und Martha Young-Scholten (Newcastle) Band 9 Ruth Trüb An Empirical Study of EFL Writing at Primary School Ruth Trüb ORCID: 0000-0002-9915-8611 Institut Primarstufe Pädagogische Hochschule FHNW Windisch, Schweiz DOI: https: / / doi.org/ 10.24053/ 9783823395430 Die vorliegende Arbeit wurde vom Institut für Bildungswissenschaften der Universität Basel im März 2021 auf Antrag der Promotionskommission von Prof. Dr. Stefan Keller, Prof. Dr. Miriam Locher und Prof. Dr. Stefanie Frisch als Dissertation angenommen. Die Druckvorstufe dieser Publikation wurde vom Schweizerischen Nationalfonds zur Förderung der wissenschaftlichen Forschung unterstützt. © 2022 · Ruth Trüb Das Werk ist eine Open Access-Publikation. Es wird unter der Creative Commons Namensnennung - Weitergabe unter gleichen Bedingungen | CC BY-SA 4.0 (https: / / creativecommons.org/ licenses/ by-sa/ 4.0/ ) veröffentlicht. Narr Francke Attempto Verlag GmbH + Co. KG Dischingerweg 5 · D-72070 Tübingen Internet: www.narr.de eMail: info@narr.de CPI books GmbH, Leck ISSN 2199-1340 ISBN 978-3-8233-8543-1 (Print) ISBN 978-3-8233-9543-0 (ePDF) ISBN 978-3-8233-0351-0 (ePub) Bibliografische Information der Deutschen Nationalbibliothek Die Deutsche Nationalbibliothek verzeichnet diese Publikation in der Deutschen Nationalbibliografie; detaillierte bibliografische Daten sind im Internet über http: / / dnb.dnb.de abrufbar. www.fsc.org MIX Papier aus verantwortungsvollen Quellen FSC ® C083411 ® www.fsc.org MIX Papier aus verantwortungsvollen Quellen FSC ® C083411 ® To Rebekka and Rahel, my two beloved goddaughters, and Elias, their brother. May your lives bear fruit abundantly! 13 15 17 1 21 1.1 21 1.2 22 1.3 24 2 29 2.1 30 2.2 30 2.2.1 30 2.2.2 32 2.2.3 34 2.3 39 2.3.1 39 2.3.2 41 2.3.3 44 2.4 46 2.4.1 46 2.4.2 47 2.4.3 48 2.4.4 49 2.4.5 51 2.4.6 52 2.5 54 2.5.1 54 2.5.2 56 Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Context of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Political and educational context . . . . . . . . . . . . . . . . . . . . . . . . Communicative and competency-based language teaching . . Swiss national standards and curriculum requirements . . . . . A Comprehensive Model of Writing Competence for Young EFL Learners Construct definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Origins of the model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hayes’ model of writing processes . . . . . . . . . . . . . . . . . Feilke’s model of literacy competence . . . . . . . . . . . . . . A comprehensive model of writing competence for young EFL learners: an overview . . . . . . . . . . . . . . . . . . Writing goals, contexts and addressees . . . . . . . . . . . . . . . . . . . Writing goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Writing in different genres . . . . . . . . . . . . . . . . . . . . . . . . Writing tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Writing processes - “doing writing” . . . . . . . . . . . . . . . . . . . . . Generating ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Activating language resources . . . . . . . . . . . . . . . . . . . . . Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Formulating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (Re-)Reading, evaluating, revising and editing . . . . . . . Writing resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Language skills and resources . . . . . . . . . . . . . . . . . . . . . Writing strategies and genre knowledge . . . . . . . . . . . . 2.5.3 58 2.5.4 59 2.6 61 2.7 62 2.7.1 63 2.7.2 63 2.8 64 2.8.1 65 2.8.2 67 2.8.3 69 2.8.4 73 2.9 82 2.9.1 84 2.9.2 88 2.10 89 2.10.1 89 2.10.2 90 2.10.3 91 3 93 3.1 93 3.1.1 93 3.1.2 94 3.2 97 3.2.1 97 3.2.2 106 3.2.3 114 3.2.4 117 3.3 118 3.4 119 3.4.1 119 3.4.2 120 3.4.3 121 Content knowledge and experience . . . . . . . . . . . . . . . . External resources: people and materials . . . . . . . . . . . . The learner’s executive and monitoring function . . . . . . . . . . Individual and external determinants . . . . . . . . . . . . . . . . . . . . Young EFL learners’ motivation to write . . . . . . . . . . . . Young EFL learners’ self-efficacy . . . . . . . . . . . . . . . . . . The writing product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pragmatic text qualities . . . . . . . . . . . . . . . . . . . . . . . . . . Sociolinguistic text qualities . . . . . . . . . . . . . . . . . . . . . . Linguistic text qualities . . . . . . . . . . . . . . . . . . . . . . . . . . . Relationship of the research findings to the CEFR . . . . Learning and change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teaching EFL writing to young learners . . . . . . . . . . . . Young EFL learners’ extracurricular use of English . . . Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part I: The young EFL learners’ writing competence . . Part II: Current teaching practices and the learners’ perception of EFL writing . . . . . . . . . . . . . . . . . . . . . . . . Part III: Predictors of the learners’ EFL writing competence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Evidence-centred assessment design . . . . . . . . . . . . . . . Data collection instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rating scales and rater training . . . . . . . . . . . . . . . . . . . . Writing tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learner and teacher questionnaires . . . . . . . . . . . . . . . . Learner interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sampling method and participants . . . . . . . . . . . . . . . . . . . . . . . Data collection and processing . . . . . . . . . . . . . . . . . . . . . . . . . . Ethical considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rating procedure, summary scoring, Rasch analysis and criterion validity of the writing scores . . . . . . . . . . . . . . 8 Contents 3.4.4 126 3.4.5 130 3.5 130 3.5.1 131 3.5.2 132 3.5.3 139 3.6 143 3.6.1 143 3.6.2 145 3.6.3 145 3.6.4 145 4 147 4.1 147 4.1.1 147 4.1.2 148 4.1.3 150 4.1.4 165 4.1.5 176 4.1.6 183 4.2 190 4.2.1 190 4.2.2 199 4.3 212 4.3.1 212 4.3.2 216 Processing of questionnaire data: scales, indices and reliability analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interview transcription . . . . . . . . . . . . . . . . . . . . . . . . . . . Standard setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selection of the standard setting method . . . . . . . . . . . . Implementation of the Body of Work standard setting method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Second implementation and final results . . . . . . . . . . . . Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Descriptive text analyses . . . . . . . . . . . . . . . . . . . . . . . . . Analysis of interview data with MAXQDA . . . . . . . . . Statistical analyses with SPSS . . . . . . . . . . . . . . . . . . . . . Multilevel analysis with MLwiN . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part I: The young EFL learners’ writing competence . . . . . . . The learners’ CEFR language level in EFL writing in grade six . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The learners’ writing competence differentiated by their future educational track at lower secondary school Text samples and detailed descriptions of text quality differentiated by CEFR language level . . . . . . . . . . . . . . Heterogeneity in young EFL learners’ texts . . . . . . . . . Communicative effect in young EFL learners’ narrative texts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Coherence in young EFL learners’ narrative texts . . . . Part II: Current teaching practices and the learners’ perception of EFL writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What teachers say about how they teach EFL writing at primary school . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The pupils’ perception of EFL writing at primary school Part III: Predictors of the learners’ EFL writing competence . Individual factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Educational factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Contents 5 223 5.1 223 5.1.1 224 5.1.2 226 5.1.3 227 5.1.4 228 5.1.5 230 5.1.6 230 5.1.7 232 5.2 233 5.2.1 234 5.2.2 235 5.2.3 235 5.2.4 236 5.2.5 237 5.2.6 237 5.3 239 5.3.1 239 5.3.2 240 5.3.3 240 243 249 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part I: The young EFL learners’ writing competence . . . . . . . The learners’ CEFR language level in EFL writing in grade six . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heterogeneity in the learners’ EFL writing competence and the transition from primary to secondary school . Characteristics of young EFL learners’ texts at different CEFR language levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heterogeneity within the learners’ texts: implications for teaching and research . . . . . . . . . . . . . . . . . . . . . . . . . Heterogeneity in the learners’ performance between different text types: implications for teaching and research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Communicative effect: a central dimension of EFL writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The importance of coherence in young EFL learners’ texts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part II: Current teaching practices and the learners’ perception of EFL writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The role of EFL writing as a communicative competence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The role of learning strategies in classroom practice . . The role of pragmatic text qualities in classroom practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The role of different genres when teaching EFL writing Elements of the process and genre approach . . . . . . . . . Motivation, resources and task demands: three important elements affecting the learners’ perception of EFL writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part III: Predictors of the learners’ EFL writing competence . The learners’ self-efficacy . . . . . . . . . . . . . . . . . . . . . . . . The learners’ extra-curricular use of English . . . . . . . . Different educational factors . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Contents 265 265 265 275 275 282 282 285 292 292 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compilation of CEFR descriptors relevant for young foreign language writers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learner questionnaire (in German) . . . . . . . . . . . . . . . . . . . . . . Appendix C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scale and index documentation: learner questionnaire . . . . . . Scale and index documentation: teacher questionnaire . . . . . . Appendix D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Detailed ratings of Profiles 1-6 . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Contents Abstract This study investigated the extent to which young learners of English as a foreign language ( EFL ) in grade 6 are able to compose texts that fulfil a communicative function. Since many teachers raised the question of how to teach EFL writing to young learners, the study also investigated current teaching practices and the learners’ perception of EFL writing, and examined different individual and educational determinants of EFL writing competence. The learners wrote two texts, an email and a story, which were rated independently by two raters. The writing scores were adjusted for task difficulty, rater severity and difficulty of the rating criteria, and aligned to the language levels of the Common European Framework of Reference for Languages ( CEFR ). Learner and teacher questionnaires and learner interviews provided additional data. The learners’ EFL writing competence ranged from below A1.1 to above A2.2, with the majority of learners at levels A1.2 and A2.1. There was a statistically significant difference in EFL writing competence between the groups of learners about to enter the different educational tracks at secondary school. In order to illustrate the learners’ EFL writing competence at different language levels, writing profiles with detailed descriptions of text quality were provided. Many texts displayed heterogeneity with regard to different dimensions of text quality. Two small qualitative analyses provided insight into the means used by young EFL learners to create a communicative effect, and into the quality of coherence in their texts. Many teachers reported using elements of the process approach, such as pre-writing activities, scaffolding and feedback. A few teachers reported that they applied elements of the genre approach, such as studying a sample text. Less frequently applied were elements such as collecting ideas on what to write about, discussing how to structure a text, or publishing the texts to a real audience. It emerged that, while pragmatic aspects (e. g. how to write a funny, sad or captivating story) are frequently considered when texts are assessed, they are only rarely addressed in class, and strategy instruction only plays a minor role in teaching practice. Three main factors were identified that appear to influence the learners’ perception of EFL writing: motivation, resources and task demands. If motivation was high, the learners had a clear idea of what they wanted to write about; if they had enough language resources and if task demands were considered as adequate, the learners showed a positive attitude towards EFL writing. The learners’ self-efficacy and extra-curricular use of English were found to be strong predictors of their EFL writing competence. None of the examined educational factors significantly contributed to the explanation of the learners’ EFL writing competence, which may have been due to limitations in the research design. 14 Abstract Acknowledgements I would like to thank the following people and institutions, without whom I would not have been able to complete this research: My supervisors Prof. Dr. Stefan Keller, Prof. Dr. Miriam Locher and Prof. Dr. Stefanie Frisch for their invaluable and continuous support and the many inspiring discussions that greatly contributed to the quality of this research project. Prof. Ursula Bader, head of our English department, who aroused my fasci‐ nation for teaching foreign languages, never ceased to support my professional development throughout the past 15 years and encouraged me to undertake this exciting journey of pursuing a doctoral degree. The FHNW University of Applied Sciences and Arts Northwestern Switzer‐ land for granting me a 20 % reduction of my workload with continued pay during three years, so that I had enough time and resources to focus on the research project and move it forward efficiently. The Aebli-Näf Foundation, Burgdorf, for its generous and repeated financial support, for which I am so grateful. My research assistants Nadia Kohler, Christina Leuenberger, Elisabeth Trüb and Lotti Strickler, who invested a lot of time, energy and diligence into this project, as well as Christian Deppeler, Shirin Lindenmann, Alice Giger and Oskar Trüb for their very helpful and greatly appreciated support. Heinz Graber for designing the model of writing competence in 3D. Dr. Kirsten Schweinberger for her perceptive feedback and advice on statis‐ tical analysis and research methodology. Marion Richner for proof-reading the thesis. The pupils and teachers who volunteered to participate in the study and opened their classrooms for this research so that, hopefully, many more classes and teachers can benefit from their knowledge and experience. The different sponsors who provided office supplies and the opportunity for the participating classes to attend a baseball workshop with English-speaking professional baseball players. The Therwil Flyers for organising the workshops. My colleagues and fellow researchers. My dear parents, family and friends, who supported me so much in all areas, who encouraged me and prayed for me. S. D. G. - Soli Deo Gloria Introduction This dissertation is concerned with the teaching of writing in English as a foreign language ( EFL ) at primary school. It gives an account of the design, implementation and results of the research project An Empirical Study of EFL Writing at Primary School, which was carried out in the Canton of Aargau, Switzerland, between 2016 and 2020. The research project investigated the writing competence of young EFL learners in grade six, when they were about 12-13 years old. Besides gauging the learners’ EFL writing competence and describing the characteristics and qualities of their texts, the study examined current teaching practices and the learners’ perception of EFL writing, and measured the effects of individual and educational factors on the learners’ EFL writing competence. So far, much of what is known about EFL writing in Switzerland stems from research in lower and upper secondary schools and from adult education. Since foreign language teaching at primary school was only introduced in the different cantons between 5 and 15 years ago, there are few studies that have researched EFL writing at primary school. Of the few existing studies, some gauged the learners’ EFL writing competence as part of a general evaluation of foreign language teaching at primary school (e. g. Bader & Schaer, 2006; Bayer & Moser, 2016; Gnos, 2012; Kreis, Williner, & Maeder, 2014), and others focused on aspects such as teacher variables (Loder Buechel, 2015), spelling (Vogt & Bader, 2017) or cross-lingual resources (Egli Cuenat, 2016). Therefore, a more comprehensive overview of the teaching of EFL writing at primary school appeared necessary. The research project An Empirical Study of EFL Writing at Primary School is a subject-specific educational study in the field of foreign language teaching research. According to Leuders (2015), subject-specific educational research is ultimately always directed towards the subject-specific teaching and learning at school and the background for the professionalisation of the pedagogical staff (p. 13-14). One of the main aims of this study was therefore to make a contribution to this professionalisation by providing the teachers with an overview of what is so far known from research about EFL writing in a young learner context. For this reason, a model of writing competence for young EFL learners has been developed (see chapter 2.2.3). The model has two main functions. Firstly, it serves as a framework for presenting the current state of research with regard to EFL writing at a young age (see chapter 2), and secondly, it forms the theoretical and empirical background on which the research project is based. If not otherwise specified, the term young learners is used to refer to pupils aged approximately 9 to 13 years. A second aim of the study was to find specific answers to questions about the young EFL learners’ writing competence. Before the project was initiated, there were many requests from teachers asking for information and guidance on how to teach EFL writing at primary school. When the first pupils who had started to learn English at primary school entered secondary school, the secondary school teachers often acknowledged that the learners had a good command of listening, speaking and reading skills, but they suggested putting more emphasis on the development of writing skills. As a result, the primary school English teachers started to ask questions about the role of writing in the primary EFL classroom and about how writing competence could be developed in an age-appropriate way. They also expressed the fear that, with a greater emphasis on writing, much of what had been achieved, including the learners’ motivation to learn languages, might be lost. Furthermore, they doubted whether a spelling approach, such as they themselves had experienced when they were taught English, would be suitable for teaching EFL writing at primary school. These questions seemed to reveal different needs, which the study tried to address. First, the questions appeared to show an uncertainty about what was meant by the term writing. Does writing mainly have to do with spelling or more with text composition? Are young EFL learners capable of writing texts, or should one limit oneself primarily to writing at word and sentence level? Such questions were addressed in two ways. First, it appeared important to provide a clear definition of writing that could be used as a starting point for the whole research project (see chapter 2.1). This definition was developed in relation to the Swiss national standards for foreign language learning ( EDK , 2011) and the cantonal curriculum ( BKS , 2018), which regard the ability to use the language for oral and written communication as the key aim of foreign language learning (see also chapters 1.2 and 1.3). Secondly, the study investigated to what extent the EFL learners in grade six are able to solve communicative writing tasks, and aimed at measuring their EFL writing competence with regard to the different language levels as stated in the Common European Framework of Reference for Languages ( CEFR , Council of Europe, 2001). An additional, descriptive analysis of the learners’ texts at the different language levels was intended to give the teachers guidance on what can be expected of young EFL learners in terms of text quality. A second aspect that emerged from the teachers’ questions seemed to be an uncertainty with regard to what teaching methodology to use and what to focus on when teaching EFL writing. Therefore, in order to lay a foundation 18 Introduction for discussing different methodological options, it was decided to additionally conduct an explorative study that investigated how EFL writing was currently taught in the primary schools in the Canton of Aargau. Since the teachers were also concerned to provide adequate writing instruction for young learners and to maintain their motivation to learn foreign languages, the study also investigated the learners’ perception of EFL writing, and tried to find out what challenges the young learners face when writing in English so that the teachers would, on this basis, be able to plan support appropriate to their learners’ age. In addition to these aims, the object of the study was to examine the effect of different individual and educational factors on the learners’ EFL writing competence in order to draw conclusions about how the development of the young learners’ EFL writing competence can be supported effectively. The study was thus divided into three parts: the young EFL learners’ writing competence (part 1), current teaching practices and the learners’ perception of EFL writing (part 2), and predictors of the young EFL learners’ writing competence (part 3). Chapter 1 of this documentation gives a brief description of the context of the study. It first describes the political and educational context, including the introduction of English as a new school subject at primary school. It then describes two approaches to language teaching that seem to have been particularly influential, namely the communicative and the competency-based approach. This is followed by an overview of the curricular requirements for foreign language teaching at primary school in Switzerland and a brief presentation of the results of four studies that gauged the learners’ EFL writing competence in grade six in different cantons. Chapter 2 starts with the specification of the construct definition of writing that underlies the whole research project. This is followed by a brief overview of the model of writing competence for young EFL learners and its origins in Hayes’ model of writing processes (2012) and Feilke’s model of literacy competence (2014). The subsequent chapters 2.3 to 2.9 describe the different elements of the model in detail and present relevant research findings from a young EFL learner context. These chapters form the theoretical and empirical background on which the study is based. Chapter 2 then concludes with the specification of the research questions. Chapter 3 presents the research methodology that was applied in the study. After a first overview of the research design, the data collection instruments, the sampling method and participants, the data collection and processing, the standard setting and the methods for data analysis are presented. The chapter describes in detail how the research methodology is linked to the construct 19 Introduction definition of writing and the empirical findings presented in chapter 2. It also discusses the quality of the research methodology in terms of validity, reliability and objectivity. Chapter 4 presents the results of the three parts of the study. It first describes the young EFL learners’ writing competence from a quantitative and qualitative perspective and gives a detailed account of the specific characteristics of the learners’ texts at different language levels. It then gives insights into current teaching practices and describes what the learners think about EFL writing. The chapter concludes with a presentation of the results of different statistical analyses that examined the effect of individual and educational factors on the learners’ EFL writing competence. Chapter 5 summarises the most important findings from chapter 4 and discusses their relevance for teaching EFL writing at primary school. It also draws conclusions with regard to EFL writing that appear to be relevant for teacher education, policy makers and research. 20 Introduction 1 Context of the Study 1.1 Political and educational context In Switzerland, English is learnt as a foreign language, i. e. it is usually neither the learners’ native language (L1), nor the official language of instruction at school, nor one of the main languages of communication in society (Richards & Schmidt, 2010). Nevertheless, global collaboration and mobility as well as technical developments such as internet technology and mobile communication networks have led to a widespread use of the English language. In most cantons in Switzerland, English is introduced as either the first or the second foreign language at primary school ( EDK , 2018). Responsibility for education in Switzerland lies with the cantons ( EDK , 2012). On an inter-cantonal level, it is the Swiss Conference of Cantonal Ministers of Education ( EDK ) who coordinate the cantonal activities by use of so-called concordats and recommendations ( EDK , 2012). After the year 2000, several cantons started to introduce English as a foreign language at primary school ( EDK , 2018), and in 2004, the EDK obtained inter-cantonal agreement that at least two foreign languages should be taught at primary school, including at least one national language ( EDK , 2004b). The introduction of foreign language learning at primary school was paral‐ leled with a shift in foreign language teaching methodology (Bader-Lehmann, 2007). In the Canton of Aargau, the teacher training for primary school EFL teachers started in 2004 and aimed at equipping the teachers and teacher trainees with solid knowledge about the current young learner EFL teaching methodology (Bader, 2006). This one-year methodology course was offered both to in-service primary school teachers and primary school teacher trainees, consisted of 90 hours contact study and 90 hours independent study and was credited with 6 ECTS points (Bader-Lehmann, 2007). In addition to the methodology course, the teachers and trainees had to stay for at least 8 weeks in an English-speaking country and were required to obtain a C1 language certificate (Bader, 2006). In 2008, after a two-year piloting phase, English was officially introduced in the Canton of Aargau as a new school subject at primary school (Husfeldt & Bader-Lehmann, 2009). In the Canton of Aargau, English is the first foreign language, and the pupils start learning it from grade 3, when they are about 9 years old. They have three English lessons a week in grades 3 and 4, and two lessons in grades 5 and 6. Grades 7-9 are taught at secondary school with three English lessons per week. It seems important to note that, by the time the pupils start learning English, they have already acquired basic literacy skills in the school language (German) and can build on this knowledge when they start learning English. It is, therefore, a different situation compared to countries where the learners start learning English in grade 1 and are simultaneously starting to develop literacy skills in the school language and in the foreign language. 1.2 Communicative and competency-based language teaching The new teaching methodology that was introduced in the teacher training fol‐ lowed the approach of communicative language teaching ( CLT ), which focuses on using the language for real communication ( BKS , 2008). In CLT , language is seen as a means of communication that fulfils a particular function (Richards & Rodgers, 2014). Language learning takes place in meaningful activities that engage the learners in real, authentic communication (Richards & Rodgers, 2014). Such a view seems to be in line with findings from second language acquisition research. McKay (2006), for example, summarising different theories from second language acquisition research, argues that in foreign and second language classrooms, children’s language learning flourishes when there is a focus on meaning, and when their teachers and other visitors give them opportunities to interact in ways that reflect the wider discourse communities relevant to the language they are learning. Children learn to use language because the interesting activities in which they are engaged absolutely necessitate (from the child’s point of view) cooperation and interaction. (p. 41) According to Richards and Rodgers (2014), learning in CLT follows three principles: the communication principle, the task principle and the meaningful‐ ness principle (p. 90). Thus, activities should include real communication, the language should be used to complete relevant tasks and the language should be meaningful to the pupils (Richards & Rodgers, 2014, p. 90). The main aim of CLT is to develop communicative competence, a term brought forward and discussed, for example, by Hymes (1972), Canale and Swain (1980) and Bachman and Palmer (1996). Even though they used slightly different specifications of this term, it generally referred to the knowledge and skills needed to be able to use the language for particular purposes (Richards & Rodgers, 2014). 22 1 Context of the Study The same term was also used with the advent of competency-based language teaching ( CBLT ). In this context, the term competencies refers to ‘the essential skills, knowledge, attitudes, and behaviors required for effective performance of a real-world task or activity’ (Richards & Rodgers, 2014, p. 156). Whereas CLT primarily focusses on the importance of meaningful communication, CBLT tries to describe the learning outcomes, what the learners can or should be able to do with the language (Richards & Rodgers, 2014). A prominent product of CBLT is the Common European Framework of Reference for Languages ( CEFR ), which defines on different proficiency levels ‘what language learners have to learn to do in order to use a language for communication and what knowledge and skills they have to develop so as to be able to act effectively’ (Council of Europe, 2001, p. 1). The two approaches had a considerable influence on curriculum develop‐ ment, teacher education and teaching practice. The standards for foreign language learning in Switzerland ( EDK , 2011) as well as the cantonal curricula (e. g. BKS , 2008, 2018) were developed on the basis of the CEFR language level descriptors and specify the competencies the learners should acquire during compulsory education at primary and lower secondary school. The curricula also emphasise the importance of communicative competence as the main aim of language learning and regard language resources as subservient to the ability to communicate effectively: The learners are to be enabled to use language skills in diverse and as authentic situations as possible. Interesting contents and factual topics from the pupils’ world of living form the basis for using the language. … For the successful completion of the communicative learning tasks, the learners need the corresponding language resources. Systematic work on vocabulary, pronunciation, grammar and orthography as well as reflection on them are described in the competence area language(s) in focus. The work on these competencies is not an end in itself but primarily serves the communicative action. [unofficial translation] (D-EDK, 2016, p. 63) The same concepts and ideas served as a basis for the new course of studies for primary English teachers at the School for Teacher Education at the University of Applied Sciences and Arts Northwestern Switzerland ( PH FHNW ). According to Bader-Lehmann (2007), holistic, communicative and action-oriented language learning should be the aim of foreign language teaching at primary school (p. 242). The English teacher trainees who attended the new course of studies were expected to study the concepts and principles of contemporary, scientifi‐ cally substantiated language teaching methodology and reflect on their own belief systems while at the same time experiencing concrete examples of these 23 1.2 Communicative and competency-based language teaching teaching practices in the seminar (Bader-Lehmann, 2007). They should thus be enabled to integrate the new concepts and methodologies into their own teaching practice, even if they had experienced a more traditional language teaching methodology in their own school career (Bader-Lehmann, 2007). When the teachers started teaching English at primary school, they were supported in small, regional professional development groups, where they had the opportunity to discuss the questions and problems that arose in everyday teaching practice (Bader-Lehmann, 2007). This seems to have had the effect that many teachers were able to implement the new teaching methodology in their classrooms. 1.3 Swiss national standards and curriculum requirements In 2011, the EDK introduced national standards for foreign language learning at primary and secondary school ( EDK , 2011). They were developed on the basis of the HarmoS validation study for foreign languages (Konsortium HarmoS Fremdsprachen, 2009) and with reference to the CEFR (Council of Europe, 2001). Also here, the underlying principle was the concept of competencyand action-oriented language learning, which regards the ability to use the language for oral and written communication as the key aim of foreign language learning ( EDK , 2011). The Swiss national standards for foreign language learning are formulated as competences that describe what the learners should be able to accomplish by the end of primary and secondary school, in grades 6 and 9 respectively ( EDK , 2011). They define key competences in listening, speaking, reading and writing as well as language mediation, intercultural awareness and the use of learning strategies. Since the CEFR levels are rather broad, the Swiss national standards further split the existing levels into two empirically validated sub-levels such as A1.1 / A1.2 and A2.1 / A2.2 ( EDK , 2012, p. 166). In terms of minimum requirements, the national standards take into account that foreign language learning at primary school is still in the process of being established. According to the EDK (2011), the standards can be raised after a transition period, when teaching methodology, teacher training and teaching materials are in place. Table 1.1 shows the target language levels for the first foreign language at the end of primary school during the transition phase and Table 1.2 the language levels the children are expected to reach after the transition phase ( EDK , 2011, p. 6). 24 1 Context of the Study CEFR language level A1.1 A1.2 A2.1 A2.2 Listening comprehension Reading comprehension Speaking (conversation) Speaking (monologue) Writing Table 1.1 Swiss national standards: Target language levels in the first foreign language at the end of primary school during the transition phase (EDK, 2011) CEFR language level A1.1 A1.2 A2.1 A2.2 Listening comprehension Reading comprehension Speaking (conversation) Speaking (monologue) Writing Table 1.2 Swiss national standards: Target language levels in the first foreign language at the end of primary school after the transition phase (EDK, 2011) As can be seen in Table 1.2, the standards are raised for listening, reading and speaking but not for writing. According to the EDK (2011), this is due to the fact that children of this age are still in the process of developing basic writing skills in the school language and because there are aims connected to writing that are not relevant for using the language. What aims they are referring to in this argument is not further specified. Following the Swiss national standards for foreign language learning, the new curriculum of the Canton of Aargau (“Lehrplan 21”) specifies the same target language levels as the Swiss national standards for the time after the transition phase ( BKS , 2018, p. 77). The learners should reach at least level A1.2 in writing by the end of grade 6: 25 1.3 Swiss national standards and curriculum requirements CEFR language level A1.1 A1.2 A2.1 A2.2 Listening Reading Speaking Writing Table 1.3 Curriculum of the Canton of Aargau: Target language level in the first foreign language at the end of primary school (BKS, 2018) The HarmoS studies, which laid the empirical foundation for setting the Swiss national standards ( EDK , 2004a), were carried out at a time when many cantons had not yet introduced foreign language learning at primary school. The consortium for foreign languages could therefore only predict what the learners might be able to achieve in the future when they would start learning English at primary school (Konsortium HarmoS Fremdsprachen, 2009). However, after the introduction of foreign language teaching at primary schools, several cantonal studies researched whether and how well the learners reached the aims set by the curriculum. Four of these studies investigated the learners’ EFL writing competence (Bader & Schaer, 2006; Bayer & Moser, 2016; Gnos, 2012; Kreis et al., 2014). All of these studies measured the learners’ EFL writing competence in grade 6, the study by Bayer and Moser (2016) at the beginning and the other three studies in the second half of the school year (Bader & Schaer, 2006; Gnos, 2012; Kreis et al., 2014). Bader and Schaer (2006) investigated the EFL learning outcomes in the Canton of Appenzell Innerrhoden. They researched whether the learners in grade 6 were able to write a simple letter, telling a pen-friend about their school. They observed large differences in terms of text length but concluded that even when the texts were very short and only consisted of three or four sentences, the learners succeeded in communicating. A link to the CEFR language levels was not established. The study in the Canton of Lucerne by Gnos (2012) used two writing tasks and an assessment grid from the measurement tool lingualevel ( BKZ , NW - EDK , & EDK -Ost, 2008) to evaluate the learners’ writing competence. They assessed the texts in terms of vocabulary, grammar, orthography and text. Gnos (2012) found that 25 % of the learners did not reach the minimum requirements, 64 % reached level A1.2 and 11 % level A2.1 or higher (see Figure 1.1, n = 650). The study in the Canton of Thurgau (n = 229) showed that 14 % of the learners did not reach the minimum requirements, 35 % reached level A1.2 and 50 % level A2.1 or higher (Kreis et al., 2014). Similarly to Gnos (2012), they used 26 1 Context of the Study tasks and assessment grids from lingualevel ( BKZ et al., 2008). However, neither of the two studies gives a detailed account of how rating reliability was ensured, how the final scores were calculated and how the cut scores between the CEFR levels were determined. Bayer and Moser (2016) give more information in this respect. They regularly monitored inter-rater reliability and adjusted the final scores for rater severity using item response theory. Experts allocated five benchmark texts to the CEFR levels, but no information is provided on how the cut scores were determined (Bayer & Moser, 2016). They found that 18 % of the 1216 participating pupils did not reach the minimum requirements, 45 % reached level A1.2 and 36 % level A2.1 and higher. Figure 1.1 EFL writing competence in different cantons (grade 6) As can be seen in Figure 1.1, the results of these studies show considerable heterogeneity between the different cantons in terms of the learners’ EFL writing competence, with 25 % ( LU ), 14 % ( TG ) and 18 % ( AG ) of the learners not yet reaching the minimum requirement (level A1.2), and 11 % ( LU ), 50 % ( TG ) and 36 % ( AG ) of the learners surpassing this level. Therefore, there seems to be a need for further research, not only for gauging the learners’ EFL writing competence, but also for giving a more detailed account of what text qualities might be expected of the learners at the different language levels, and for what factors might influence the learners’ writing competence. These are all questions the research project tried to address (see chapter 2.10, research questions). 27 1.3 Swiss national standards and curriculum requirements 2 A Comprehensive Model of Writing Competence for Young EFL Learners If writing competences are to be gauged for research purposes, a solid theoretical model appears central. It lays the foundation for any decisions that are to be taken in terms of research methodology, and provides a framework for interpreting the results. Chapter 2 presents a model of writing competence that has been specifically developed for a young EFL learner context, and which forms the basis for this research project. As already mentioned in the introduction, the study deals with various facets of learning to write in English as a foreign language at primary school. It deals with questions of writing competence, text quality, learning and teaching as well as predictors of writing competence. These aspects concern closely interwoven, complex processes that cannot be explained in a simple way. The study therefore requires a model that is capable of representing the different facets of writing and writing competence and their relationships to each other, while providing an intuitively comprehensible framework within which the results of the study can be discussed. As far as possible, the model of writing competence is based on empirical evidence from a young EFL learner context. If no or only little evidence from this context was available, or if findings from related fields and contexts appeared central, for example from a young L1 learner context, such further findings were added to the discussion. The main aim of the study, however, is not to validate the model. This is something further research studies may be able to contribute to. The model of writing competence for young EFL learners is used as a framework for presenting the theoretical and empirical background of the study and for discussing the different elements of the research design. Moreover, it serves as a basis for presenting and discussing the results of the study. Chapter 2.1 first presents the construct definition of writing on which the model is based. Chapter 2.2 briefly describes its origins and gives a first overview of the model. Chapters 2.3 to 2.9 present the different elements of the model in more detail, discussing various research findings and theoretical considera‐ tions that appear central for understanding the particular characteristics and challenges of writing in a young EFL learner context. References are provided to the corresponding sections in chapter 3, which show how these considerations have been integrated in the research methodology. The chapter concludes by specifying the research questions of the study (chapter 2.10). 2.1 Construct definition The model of writing competence for young EFL learners presented in this chapter was developed on the basis of a communicative and competency-based view of language learning as presented in chapter 1.2. Writing is regarded as a complex individual or collaborative activity that leads to the creation of a written product that fulfils a particular function. The final product may display different pragmatic, sociolinguistic and linguistic qualities. Writing competence is defined as the writer’s ability to use his or her personal and external resources in order to effectively and responsibly perform real-world writing tasks. The writing process may - to varying degrees - be influenced by diverse personal and external factors which may either support or hinder the writer’s performance. Learning and change is an important aspect of the model, since it allows writing development to be seen in its multidimensionality, and teaching approaches can be discussed with regard to the different elements of the model they focus on. Before the model is presented in detail, the chapter now gives a brief outline of its origins in Hayes’ model of writing processes (2012) and Feilke’s model of literacy competence (2014). 2.2 Origins of the model 2.2.1 Hayes’ model of writing processes One of the most prominent and comprehensive models of writing processes is the model by Hayes (2012), see Figure 2.1. It originates from a model presented by Hayes and Flower (1980), which was modified several times over the years. It was designed on the basis of what is known from research and should, according to Hayes and Olinghouse (2015), ‘be viewed as a work in progress rather than as a finished product’ (p. 482). Even though its intention is to describe adult writing (Hayes & Olinghouse, 2015) rather than writing in a young EFL learner context, it encompasses important elements that also seem to be relevant in a young learner context. The model describes the processes involved in writing and consists of a control, process and resource level (Hayes & Olinghouse, 2015). The control level includes aspects that direct the writing activity, namely the motivation to write, goal setting, the current writing plan and writing schemata such as genre knowledge and writing strategies (Hayes & Olinghouse, 2015). The process level comprises the actual writing process when ideas are created (proposer), 30 2 A Comprehensive Model of Writing Competence for Young EFL Learners mentally turned into language (translator) and transformed into written text (transcriber) (Hayes & Olinghouse, 2015). The evaluator judges the adequacy of ideas, mental language and written text (Hayes & Olinghouse, 2015). Also part of this writing process (but not labelled in the model, since they are complex activities that make use of the basic processes just described) are planning and revision (Hayes & Olinghouse, 2015). Moreover, the social and physical task environment, such as feedback from peers or the task materials, is believed to have an influence on the writing process and is, therefore, allocated to the process level (Hayes & Olinghouse, 2015). The third and last level of the model is the resource level, which includes aspects such as the ability to focus attention, the long-term and working memory, and reading as an important and recursive element that is frequently drawn upon during the writing process (Hayes & Olinghouse, 2015). Figure 2.1 Model of writing processes by Hayes (2012, p. 371) Such a model is helpful for understanding the different cognitive processes involved in writing. It shows that writing not only concerns the act of putting words on paper but is a highly complex mental activity. Hayes’ writing model, however, is mainly a model of cognitive processes rather than a model of writing competence, and has been criticised for taking too little account of the language resources required for successful writing (see e. g. Bachmann & 31 2.2 Origins of the model Becker-Mrotzek, 2017). Even though Hayes and Olinghouse (2015) acknowledge that children need to build up language resources and writing schemata such as genre knowledge and strategies, it seems crucial to give these aspects a more prominent place in a writing model if it is to be applied to a young EFL learner context, where two of the main challenges of writing appear to be the learners’ limited language resources and their comparably small writing expertise. 2.2.2 Feilke’s model of literacy competence A model that takes these aspects into account is Feilke’s model of literacy competence (2014), see Table 2.1. He defines literacy competence (literale Kompetenz) as the ability of an individual to participate in the literary practices of a particular culture through the reception and production of written texts according to the expectations of that culture (Feilke, 2014, p. 43). Competence Type of acquisition Relevant conditions Control level “What for”- Compe‐ tences Cultural Competences ● Value orientation ● Knowledge of norms ● Motivation ● Literacy-related target con‐ cepts ● Literacy-related task sche‐ mata ● Literacy-related role concepts and practices Socialisation Enculturation Literacy socialisation in the family (e. g. early orientation towards literacy, models of literacy actions, genderspecific aspects of writing and reading) Secondary and tertiary socialisation (school, religion, science, law, media) ● Domain-specific literacy practices and corresponding models of literacy actions ● Canonically marked cultural inven‐ tories (text canons, norms of written language) & competence standards ● School instruction, e. g. tasks Process level “How”- Compe‐ tences Writing competences ● Writing-reading routines ● Planning strategies ● Writing strategies ● Revising strategies Text handling competences ● Text composition ● Formulation and reformula‐ tion procedures ● Paraphrasing procedures ● Compacting procedures Problemsolving actions ● Exploratory and self-controlled reading and writing ● Attention to the process and reflec‐ tion on the writing act ● Inclusion of writing ecology factors (addressee feedback) ● Process-related instructional scaf‐ folding (e. g. through temporary sus‐ pension of norms) 32 2 A Comprehensive Model of Writing Competence for Young EFL Learners Competence Type of acquisition Relevant conditions Resource level “What”- Compe‐ tences Acquired resources (LTM) Literacy-related language com‐ petences: ● Genre knowledge ● Knowledge about text-proce‐ dural routines ● Sentence and phrase construc‐ tion ● Literacy-related lexicon Orthographic lexicon and norm competence Declarative and episodic knowl‐ edge of the world (terms, hypoth‐ eses, frames & scripts) Acquisition of reading and writing skills Concept-/ hypothesis-forming learning ● Experience with reading and writing in different domains ● Sufficient writing and reading times (“time on task”) ● Teaching resources prepared to be used as examples (materials, expect‐ ations in terms of school language, forms and practices etc.) ● Attention to the language & reflec‐ tive practices with a focus on lan‐ guage ● Experience of the world, processing of language, cognitively stimulating communication Constitutional resources ● Working memory ● Intelligence independent of language Table 2.1 Levels of literacy competence by Feilke (2014, p. 50) [unofficial translation] Feilke’s model (2014) encompasses the same levels as Hayes’ model, namely the control, process and resource level, but it clearly gives more importance to the language resources and the cultural context. It explicitly states language-related resources such as genre knowledge, knowledge about text-procedural routines, sentence and phrase construction and a literacy-related lexicon on the resource level (see Table 2.1, resource level). In Hayes’ model, these language-related resources are, together with further aspects such as factual or experiential knowledge, subsumed under the term long-term memory (Hayes & Olinghouse, 2015, p. 486), and therefore hidden at first sight. A second aspect which is more prominently displayed in Feilke’s model is the cultural aspect on the control level (see Table 2.1, control level). In terms of competence, it encompasses elements such as values, knowledge of norms or literacy-related schemata, and in terms of acquisition, it relates to the socialisation, e. g. in the family, at school or in society in general (see Table 2.1, competence and acquisition). Generally, Feilke’s model appears to relate more closely to an educational context than Hayes’ model, since it includes relevant conditions which might be worthwhile to consider in an instructional setting (see Table 2.1, relevant conditions). Bearing in mind that Feilke (2014) describes his model as necessarily selective and perspectival (p. 49), there appear to be certain key elements of writing and writing competence which are not or only marginally considered in this model. First, there is no explicit mention of the writing product, which 33 2.2 Origins of the model is the actual goal of the writing process, the means by which a particular communicative function is to be achieved, and a central element when it comes to measuring writing competence (see e. g. Grotjahn, 2017). In addition, the resource level mainly focuses on internal personal resources and only slightly touches on external resources that might be used by the learner during the writing process. Feilke (2014) mentions teaching resources that are prepared to be used as examples (see Table 2.1, resource level). However, elements such as collaborating peers, the teacher, print or digital resources, transcription tools such as pens, pencils, computer keyboards or also technology such as word processing software or multimedia applications are not considered in the model. Furthermore, only a few personal determinants such as values or motivation are mentioned, and no external determinants such as the influence of noise, distraction, temperature or light quality. These elements all seem to be important components and determinants of writing and are thus included in the model of writing competence for young EFL learners as presented in the following chapter. 2.2.3 A comprehensive model of writing competence for young EFL learners: an overview The model of writing competence for young EFL learners presented in this and the following chapters takes up various elements of Hayes’ (2012) and Feilke’s (2014) models (see chapters 2.2.1 and 2.2.2) but rearranges and extends them in order to allow for a more comprehensive view of writing competence. Similarly to these two models, it comprises three levels, whose contents, however, have been slightly altered: the target level, the performance-product level and the resource level (see Figure 2.2). Additionally, the model explicates the central role of the learner in his or her executive and monitoring function. Rather than referring to impersonal expressions such as proposer, translator or transcriber (Hayes, 2012, p. 371), which may evoke associations with technical devices that are placed in the learners’ brains, the model emphasises the active and creative role of the learner. The model also encompasses personal and external determinants that may have an influence on the learner’s executive and monitoring function. In order to provide an overview, these elements will now be briefly outlined, followed by an illustration of the non-static character of the model and a brief introduction to the role of learning and change. In the subsequent chapters (chapters 2.3-2.9), the model will then be discussed in more detail. 34 2 A Comprehensive Model of Writing Competence for Young EFL Learners Figure 2.2 A comprehensive model of writing competence for young EFL learners 35 2.2 Origins of the model The target level of the model of writing competence for young EFL learners describes different elements that constitute the aim of a writing task. This aim may be set by the writer himor herself, by another person or by an event that triggers a written reaction. Different elements, such as the purpose and function of the writing activity, the intended reader, the characteristics of the writing task, the content to be included, genre-specific and cultural conventions and also formal requirements, may shape the appearance and content of the intended end product. In addition to the product goals, there might be certain process goals that are pursued, such as the use of specific writing strategies. The performance-product level describes the actual writing activity and outcome. The performance consists of both internal and observable actions. It involves the typical elements of the writing process such as generating ideas, activating resources, planning, formulating, writing, (re-)reading, evaluating and revising / editing the text. If the product is a multimodal or digital text, this process may include further elements such as creating graphics and hyperlinks, drawing illustrations, taking and inserting photos, recording and editing audio or video files and similar elements. The writing activity leads to the product. At first, it may be an incomplete text or draft, later a finalised product. This product may have different pragmatic, sociolinguistic and linguistic qualities that might be analysed for assessment purposes, or for assigning the learner’s performance to a certain language or performance level. The resource level comprises the internal and external resources the writer may draw upon during the writing process. The internal resources might be stored in either the writer’s long-term or short-term memory. The learner may, for example, use vocabulary that is already stored in long-term memory, or apply a writing strategy the class has just discussed, and is currently stored in the short-term memory. The internal resources may encompass language skills and resources in first, second and foreign languages, writing strategies and genre knowledge. Furthermore, they might embrace knowledge of the content and experience, and different skills such as motor, cognitive, social and technology skills. In addition to these internal resources, external resources might be available such as people (e. g. parents, teachers or peers), materials (e. g. paper, pencils, books or dictionaries) or technology (e. g. computers, internet access or different types of software). The writer in their executive and monitoring function is a further key element of the model and encompasses the performance level and all physical and mental activity that contributes to the writing process. It consists of the learner’s ability to access, process and combine information, take action, focus and maintain 36 2 A Comprehensive Model of Writing Competence for Young EFL Learners attention as well as monitor and evaluate progress and influencing factors, and is closely linked to and at times restricted by the writer’s working memory. Besides these elements, different personal and external determinants might affect the learners’ executive and monitoring function and, consequently, have an influence on the writing process and the outcome. Personal factors might be the writer’s motivation, self-efficacy, attitude, volition, emotions, health, energy level and similar matters. In this model, these aspects are mentioned separately from the writer’s internal resources, since they appear to affect the writer directly in their executive and monitoring function. They seem to play a more active role than resources that are accessed and used. This distinction, however, may be disputed. External factors such as distraction, noise, temperature, light and air quality and similar aspects may also affect the writer in his or her executive and monitoring function. The whole model should not be interpreted as a static model. It might best be displayed in an animated form that illustrates which elements of the model are active to what extent at different points during the writing process. A learner, for example, may first be inspired by the idea of writing a letter to her English-speaking grandparents to invite them to a school theatre event (target level: purpose). She may think about what content to include in her letter (target level: content), thus generating first ideas (performance level: generating ideas) based on her knowledge about the event, such as time and place (internal resources: knowledge of the content), and activate language resources that might be suitable for this task (performance level: activating resources and internal resources: L2 language skills and resources). Since she is so eager to invite her grandparents (personal determinants: motivation), she may simply skip the planning phase (performance level: planning) and directly start to formulate her ideas and write them down (performance level: formulating and writing). She may remember that she should start a letter with a greeting (internal resources: genre knowledge and target level: genre-specific conventions), and activate vocabulary and phrases from her long-term memory (internal resources: L2 language resources). She might look up certain words in a dictionary (internal resources: writing strategies and external resources: materials) and ask her parents for support (external resources: people). She may get stuck because she does not know a particular English expression (internal resources: L2 language resources), cannot find it in the dictionary (external resources: materials) and cannot ask her mother because she is on the phone (external resources: people). She might, therefore, decide to write this phrase in the school language (internal resources: L1 language resources). She may reread (performance level: rereading) what she has written so far (product level), 37 2.2 Origins of the model evaluate its appropriateness (performance level: evaluating) based on what she knows about writing letters (resource level: genre knowledge) and considering what her grandparents may need to know in order to come to the event (target level: reader). She may try to revise and edit her draft (performance level: revising and editing) but realise that she cannot concentrate any more (executive and monitoring function) because her siblings are playing hide-and-seek in the garden (external determinants: distraction) and she is getting tired (personal determinants: energy level). For this reason, she might hand over her text to her mother for feedback (external resources: people) and decide to continue her work on the next day. This is a brief and simplified illustration of how closely the different elements of the model are interlinked. As can be seen from this example, writing does not follow a pre-defined sequence, but appears to be a highly individual and complex activity. Writing competence is thus seen as the learner’s ability to act in situations like these in order to create a written product that fulfils a particular function. Even more complexity comes into play when the aspect of learning and change is considered. In a school context, the teachers may exert influence on different aspects of writing in order to support the learners in developing their writing competence. They may, for example, define the target level by assigning specific writing tasks; they may support the learners on the performance level by helping them to plan their texts or by giving feedback on the product; they may provide scaffolding in the form of language resources or help the learners build up content and genre knowledge; they may teach writing strategies to help the learners focus their attention on relevant aspects during the writing process; they may try to control external determinants such as noise and distraction; or they may try to positively influence the learners’ motivation and attitude towards writing. The learning seems to be as varied as the teaching and a multidimensional process that may similarly concern the various aspects of writing as presented in the model. After this overview, the following chapters examine the different levels and elements of the model of writing competence for young EFL learners in more detail. They present and discuss relevant research findings from a young EFL learner context in order to lay a solid basis for understanding the specific characteristics and challenges of writing in this context. They form the theoretical and empirical background for the design and implementation of the study (see chapter 3), and are used as a framework for the discussion and interpretation of the results (see chapters 4 and 5). 38 2 A Comprehensive Model of Writing Competence for Young EFL Learners 2.3 Writing goals, contexts and addressees 2.3.1 Writing goals The target level of the writing model describes the different elements which constitute the aim of a particular writing task. This aim may be set by the learner himor herself, by another person such as the teacher or by a particular incident that requires a person to react in written form. A learner, for example, may want to invite someone to a school theatre event, a teacher may assign a particular writing task to the class, or a present may prompt someone to write a letter of thanks. In detail, the target level defines what function a text should fulfil, to whom it is addressed, what it should look like and contain, and what requirements it should meet. Additionally, the target level may include process goals the writer may want or have to pursue, such as the use of specific writing strategies in the context of strategy instruction, or agreements concerning collaboration. Donovan and Smolkin (2002) report on elementary school children who had clear ideas about the purpose and the intended readership of their texts. Betsy, for example, a first grader, explained, ‘I just wanted them … to know that I know about parrots because I have four birds’ (p. 462-463). Debra, a third grader, argued that beginning a story with ‘once upon a time’ was too ordinary, and Frieda, a fifth grade learner, declared that writers should try to catch the reader’s attention right at the beginning of a story (Donovan & Smolkin, 2002, p. 462). Frank, another fifth grade learner, ‘wanted his writing to inform his audience … about the ingenious aspects of Eskimo culture, and he sought to have them see Eskimos as he did’ (Donovan & Smolkin, 2002, p. 462). Therefore, it does not seem to be too demanding a task to discuss the goals of writing with young learners, at least if it is done in a simple and age-appropriate way. The target level of a writing task may be dealt with in particular at the beginning of the writing process, when the writer plans the final product and the intended process, but it may also be referred to while writing and when revising, editing, evaluating or assessing the text. In terms of young learners, Rijlaarsdam and Van den Bergh (2006) in a study among 15-year-old, and van der Hoeven (1997) in a study among 12-year-old L1 learners, found that the activity of reading the task assignment usually occurred at the beginning of the writing process. This, however, was not always the case. Learners who scarcely read the assignment at the beginning, or mainly did so towards the end of the writing process, produced texts of lower quality than learners who studied the assignment when they started to write (Rijlaarsdam & Van den Bergh, 2006, 39 2.3 Writing goals, contexts and addressees p. 43). Therefore, it seems important to support the learners in thinking about the target level of a writing task at the very beginning, before they start to write, and to give them enough time to study the task assignment and clarify questions (see chapter 3.2.2, instructions, for information how this was implemented in the study). Graham, McKeown, Kiuhara, and Harris (2012) and Schunk and Swartz (1991) similarly found that goal setting has a positive effect on writing quality. Graham et al. (2012), in a meta-analysis of writing instruction at elementary school, found that setting specific product goals significantly improved writing quality ( ES = 0.76). Schunk and Swartz (1991) researched the use of process goals among L1 learners at elementary school. The pupils were instructed how to use a particular writing strategy and additionally given different types of goals (Schunk & Swartz, 1991). If goal setting was specifically related to the use of the writing strategy, the learners showed higher writing skills compared to learners who had received the same type of instruction but been given the rather general product goal ‘You’ll be trying to write a descriptive paragraph’ (Schunk & Swartz, 1991, p. 13). Schunk and Swartz (1991) additionally found that goal setting was most effective when the learners received periodical feedback on their progress. In this case, not only the learners’ writing skills, but also their self-efficacy and their perception of learning the strategy, were higher than in the control groups (Schunk & Swartz, 1991). Furthermore, Midgette, Haria, and MacArthur (2008) found that providing fifth and eighth grade learners with specific goals for improving content and reader-orientation when revising texts resulted in higher text quality than setting more general and unspecific goals. Therefore, it appears central that writing goals should be specific, so that the learners clearly know what to focus on, and that the learners receive periodical feedback on how they progress. This seems to be particularly relevant for less competent writers. Griva et al. (2009), who researched the composing process of young EFL learners aged 12 in Greece, observed that less competent writers displayed considerably more difficulties in focussing their attention on specific aspects than more competent writers did. When revising their texts, for example, less able writers revised them only superficially and many of them ‘got distracted by punctuation, formatting and they were often overwhelmed - discouraged by the demands of writing’ (Griva et al., 2009, p. 141). Thus, the setting of specific and manageable product and / or process goals may help the learners focus their attention on relevant aspects. Moreover, authentic goals and contexts may also be key elements in fostering the learners’ engagement and in increasing their motivation to write (Bruning & 40 2 A Comprehensive Model of Writing Competence for Young EFL Learners Horn, 2000). Bruning and Horn (2000), for example, emphasise the importance of ‘tasks that generate engagement through their intrinsic qualities and require a minimum of externally managed rewards to keep students involved’ (pp. 27-28). Therefore, the setting of relevant, motivating and achievable goals appears to be a key element when helping young EFL learners to develop their writing skills (see chapter 3.2.2, instructions, for how the writing goals were specified in this study). 2.3.2 Writing in different genres A concept that combines different elements of the target level is that of genres. They are defined as ‘abstract, socially recognised ways of using language to achieve particular purposes’ (Hyland, 2019, p. 18). They comprise a shared understanding among the members of a community about the characteristics of a particular text or discourse type (Hyland, 2007, p. 149) and, therefore, shape much of what is defined as the target of a writing task. A story, for example, may usually be regarded as a piece of text that is intended to entertain the reader and includes elements such as the setting of the scene, a complication, a resolution and a story ending, and might often be written in the past tense and contain direct speech and emotions (see e. g. Augst, Disselhoff, Henrich, Pohl, & Völzing, 2007). The term genre, however, is not only limited to written texts. According to Hallet (2016), a genre can be presented in different modes (e. g. oral, written, multimodal or digital). He distinguishes between the three macro genres narratives, expository texts and arguments, and lists several distinct genres in each category (see Table 2.2). What appears to be missing in this list, at least at first sight, is the aspect of correspondence, for example in the form of an e-mail or a letter. Hallet (2016), however, argues that these are micro genres that contain or are dominated by elements of the key genres. A letter, for example, could be an anecdote (telling a friend about an incident at a holiday camp) or an exposition (a letter to the president about the idea of abolishing child labour) (Hallet, 2016, pp. 82, 86). Similarly, school magazine articles, blog entries or flyers may all contain different elements of the genres presented in Table 2.2. Nevertheless, it appears important to notice that these micro genres often have specific requirements and demand a certain type of language, structure or layout that are specific to that particular textual form. In correspondence, for example, these might be aspects of how to address the reader directly or of how to start and end a letter, which are elements that do not explicitly appear in the genres presented in Table 2.2. Chapters 3.2.1 and 3.2.2 provide information on how such genre-specific 41 2.3 Writing goals, contexts and addressees elements were included in the rating scales and tasks used for estimating the learners’ writing competence, and chapters 4.1.3, 4.1.5 and 4.1.6 report on how the young EFL learners in this study applied genre-specific elements in their writings. Table 2.2 Genres for foreign language teaching by Hallet (2016, p. 78) Where young EFL learners are concerned, the question arises of whether all genres are relevant for them, or whether some may have to be given preference over others. The curriculum of the Canton of Aargau ( BKS , 2018) as well as the collated representative samples of CEFR descriptors for young learners (Szabo, 2018) mainly refer to expository and narrative texts on the levels A1 and A2, such as descriptions, reports, recounts and simple stories. In addition, they explicitly state that the learners should be able to compose different types of correspondence such as text messages, greeting cards, e-mails or letters in order to give or ask for information, express thanks, apologise, invite someone or respond to an invitation, confirm or change an arrangement and wish someone a happy birthday ( BKS , 2018; Szabo, 2018). Argumentative texts are first mentioned at level B1 in the curriculum ( BKS , 2018) and at level A2+ in Szabo (2018). Hallet (2016), however, argues that the concrete realisation of a genre and the communicative activity associated with it are not bound to 42 2 A Comprehensive Model of Writing Competence for Young EFL Learners fully developed foreign language skills (p. 90). He emphasises that the pupils should be encouraged from the very beginning of foreign language learning to produce meaningful written messages in social interactional situations. In foreign language classes at primary school, this might be, for example, a very simple three-sentence-narrative or a two-sentence argument, and could, over the years, develop into more elaborate forms of written products at secondary school (Hallet, 2016, p. 70). Therefore, even though the curriculum and the CEFR descriptors refer to a rather narrow range of genres at the levels A1 and A2, it might be possible to expand this range and also include, for example, very simple forms of personal arguments, explanations, procedures or visions (see Table 2.2). Furger and Lindauer (2013) as well as Bouwer, Béguin, Sanders, and van den Bergh (2014), in studies among L1 primary school learners, found that the children’s writing skills differed depending on the genre employed. This appears to be of particular importance when gauging the learners’ writing competence, since the selection of the genre may have an influence on the final results. The authors recommend that several tasks from different genres be set when measuring the learners’ writing skills, in order to avoid a bias by genre (Bouwer et al., 2014; Furger & Lindauer, 2013). How this recommendation was taken into account in this study, and what kind of genres were selected, is presented in chapter 3.2.2. According to Hallet (2016), genres can serve as models for language produc‐ tion and social interaction (p. 104) and, thus, give the learners guidance, and help them understand the specific requirements of a particular genre. This appears to be particularly important in a young EFL learner context, where the learners may not yet have much writing expertise and only limited language resources, and might thus need more support than more experienced learners. The use of genres to specify the target of a writing task, however, may also mean that there is a danger of restricting the learners’ creativity because of conformity, or of exclusively focusing on genres that reflect the prevailing culture (Hyland, 2011, pp. 25-26). Hence, it appears central to find a good balance between using genres as an instructional scaffolding and providing opportunities to express creativity. Chapter 3.2.2 shows how this balance was sought in the writing tasks used for this study. The topic of genres will be resumed in chapter 2.5.2, where it is discussed from the perspective of writing resources. 43 2.3 Writing goals, contexts and addressees 2.3.3 Writing tasks The genre alone, however, does not fully specify the target level of writing. A further key element appears to be the writing task, in particular in instructional settings, where tasks are often planned by the teachers rather than the learners themselves. Writing tasks specify the target level in more detail than genres and give the learners specific information on what they are expected to do. Information on the topic and content, the intended audience, the type of transcription tools that can be used or requirements in terms of language or layout, are only some examples of what a writing task may specify. A first key issue when developing writing tasks is that the tasks should reflect the construct definition of writing (Grotjahn & Kleppin, 2017a). Depending on how writing is defined, different types of writing tasks might be used (Grotjahn, 2017). If writing is defined as the ability to accurately transcribe oral messages, the writing task might be a dictation. If writing is defined as the ability to write specific words, the learners may have to label different pictures. If writing is defined as the ability to use writing in communicative situations and for a particular purpose, the writing task may be, for example, to write a letter to a friend or to write a story. Chapter 3.2.2 describes the tasks that were used in this study to operationalise the construct definition of writing as presented in chapter 2.1. Another central issue appears to be the question of what might need to be considered when developing writing tasks for young EFL learners. Compared to adult or more experienced learners as well as L1 learners, young EFL learners appear to have different preconditions that may need to be considered when developing writing tasks. First, young EFL learners may have a different language level than L1 and more experienced learners. According to different studies, the learners’ language level in EFL writing at primary school ranges between the levels A1 and A2 in Switzerland (see Figure 1.1). Writing at these levels may entail writing simple phrases and sentences at level A1 and a series of simple phrases and sentences, as well as simple texts at level A2 (see Appendix A, Table A1). Writing tasks for young EFL learners should, therefore, aim at these levels and give both more and less skilled writers the opportunity to be successful. This seems to be particularly important since young learners face many years of classroom lessons and it is important that they feel, and are, successful from the start. Too many demands early on will make them anxious and fearful of the foreign language; too few demands will make language 44 2 A Comprehensive Model of Writing Competence for Young EFL Learners learning seem boring. Careful selection and grading of goals is one of the key tools available to teachers to build success into learning. (Cameron, 2001, p. 29) Secondly, it appears crucial that writing tasks should be meaningful and interesting and engage the learners in purposeful communication. According to McKay (2006), second language acquisition research shows that in foreign and second language classrooms, children’s language learning flourishes when there is a focus on meaning, and when their teachers and other visitors give them opportunities to interact in ways that reflect the wider discourse communities relevant to the language they are learning. Children learn to use language because the interesting activities in which they are engaged absolutely necessitate (from the child’s point of view) cooperation and interaction. (p. 41) According to Cameron (2001), tasks for young EFL learners should also be ap‐ propriate for their age and socio-cultural experience and focus on language that will provide a solid basis for more demanding tasks in later years (pp. 30-31), and thus focus on familiar topics, everyday experiences and personal interests. This appears to be particularly important since the learners may not yet have an extensive vocabulary for writing about a wide range of topics. Furthermore, it appears central that careful consideration should be given to how the instructions for writing tasks are formulated and presented. Cho and So (2014) researched young EFL learners’ perception of test questions and task descriptions in order to find construct-irrelevant factors that might influence the learners’ performance. They found that aspects such as complex language or ambiguity in visuals and task descriptions had a negative impact on how well the learners understood the instructions. Cameron (2001) similarly argues that we should also notice the important point that making very small changes to the information (adding dates) or to the activity (using separated pictures singly) can lead to very large changes in the tasks as experienced by pupils. This is a very powerful tool: if teachers have repertoires of such small changes, they can use them to adapt and adjust tasks found in course books to suit particular learners. (p. 35) In summary, the writing task appears to be one of the key elements for specifying the target level of writing. As such, the writing task seems to have a considerable influence on the entire writing process and the final product. In a young EFL learner context, developers of writing tasks may have to consider the specific prerequisites of young foreign language learners, namely their language level, the importance of reaching a sense of achievement, the central role of meaningful, communicative tasks for the progress of children’s learning, their 45 2.3 Writing goals, contexts and addressees need for familiarity with topics and language, and their need for clear and simple instructions. Chapter 3.2.2 describes in detail how these prerequisites of young foreign language learners were considered in the development of the writing tasks for this project. 2.4 Writing processes - “doing writing” The performance level of the model of writing competence for young EFL learners (see Figure 2.2) describes the actual writing activity. It consists of the observable and mental actions that lead to the final product such as gen‐ erating ideas, activating (language) resources, planning, formulating, writing, (re-)reading, evaluating, revising and editing the text. These elements should not be regarded as a linear sequence of events, but rather as ‘recursive, interactive and potentially simultaneous’ (Hyland, 2019, p. 11). This chapter summarises empirical findings on the writing processes of young EFL learners when composing texts. Since many teaching practices are related to the different writing processes, these considerations form part of the empirical background for the investigation of the current teaching practices, and of the learners’ perception of EFL writing at primary school (see chapters 4.2.1 and 4.2.2). 2.4.1 Generating ideas Generating ideas refers to the process of thinking about, and making suggestions for, what to include in a text. According to Hayes and Olinghouse (2015) these ideas ‘may be triggered by the sensory environment, by memory, by goals, by collaborators, by source texts (either paper or electronic), and by what the writer has written so far’ (pp. 483-484). Griva et al. (2009) researched the writing processes of young EFL learners aged 12 in Greece. In terms of idea generation, the majority of learners reported that they would frequently draw upon their prior knowledge (knowledge about content, language and context) to generate ideas for writing. Stronger learners were found to do this more frequently than intermediate and low-level learners (Griva et al., 2009, p. 136). Some learners, in particular less able writers, regarded brainstorming with peers on the topic as an important strategy (Griva et al., 2009). Less able writers were found to generate less content than stronger learners did (Griva et al., 2009). In a different study among 10-12 year-old pupils learning Greek as a second and English as a foreign language, Griva and Chostelidou (2013) found that most of the stronger pupils relied on their 46 2 A Comprehensive Model of Writing Competence for Young EFL Learners prior experience and on external resources when generating ideas for text composition in English. Regardless of their language level, the majority of pupils also used the title to generate ideas (Griva & Chostelidou, 2013). Griva and Chostelidou (2013) additionally analysed whether there was a statistically significant difference between learners with high and low writing abilities in terms of how efficiently they generated ideas for EFL writing. They found a highly significant difference, with 94 % of the strong learners and only 13 % of the less able writers generating ideas in an efficient way (Griva & Chostelidou, 2013, p. 3). These findings seem to indicate that considerable differences between highand low-ability learners already exist at the stage of generating ideas. Less able learners, in particular, appear to appreciate the support of peers when generating ideas. Research on the effectiveness of activities to support this process (e. g. by discussing the topic in class, brainstorming ideas or creating a mind map) is usually conducted in the context of planning writing, and therefore discussed in chapter 2.4.3. 2.4.2 Activating language resources Activating language resources is regarded as the writer’s conscious effort to search for language he or she may need during the writing process. While an experienced L1 writer may not or only rarely have to consciously search for missing language before or while writing, this seems to be one of the central challenges for young EFL learners, whose language resources are limited. Griva and Chostelidou (2013), in their study among young learners who learned Greek as a second and English as a foreign language, found that only 6 % of the learners with high writing abilities recalled vocabulary before they started to write in Greek, but 75 % of the less able writers did so (p. 3). When writing in English, no statistical difference between highand low-ability learners could be found because both groups applied this strategy (Griva & Chostelidou, 2013, p. 3). Thus, in a young learner context, less able second language writers as well as most foreign language learners appear to rely on activating language resources before starting to write. Therefore, it appeared central to explicitly add this aspect to the model of writing competence for young EFL learners as presented in chapter 2.2.3 (see Figure 2.2). Activating language resources, however, seems to be a key aspect that takes place not only before the actual writing activity, but also during the writing process. At this stage, it appears to be closely linked to formulating, i. e. the 47 2.4 Writing processes - “doing writing” process of transforming ideas into language. Therefore, the aspect of activating language resources will once again be considered in chapter 2.4.4. Besides language resources, the learners may activate various other internal and external resources at the beginning and during the writing process. These aspects will be discussed in chapter 2.5. 2.4.3 Planning Planning is regarded as the process of creating a mental or written plan of the writing product and / or process. If it is a mental plan, it may be stored in the learners’ shortor long-term memory (see Figure 2.2, internal resources); if it is a written plan, it becomes part of the learners’ external resources (see Figure 2.2, external resources). Such a plan may relate to various elements of the target, product, performance and resource level, and also encompass the learners’ intentions on how to deal with personal and external determinants that may affect their executive and monitoring function. In an L1 context, Bereiter and Scardamalia (1987) found that children tend not to plan their texts before they start writing unless they are instructed and supported to do so, or only do so briefly. Furthermore, they found that some children struggled to distinguish between planning and composing and directly started to write when they were asked to create a plan. These findings were confirmed by Olinghouse and Graham (2009) who found in a study among second and fourth grade pupils that about two thirds of the participants did not plan their texts, or only scarcely planned them (p. 43). In a young EFL learner context, Griva et al. (2009) found that more than half of the learners tried to engage in some sort of planning and generating ideas before writing (Griva et al., 2009). About two thirds of the pupils reported that they tried to generate first ideas, select appropriate vocabulary and use their prior knowledge for planning (Griva et al., 2009). 19 % of the learners indicated that they would often consider the purpose of the writing when planning it (Griva et al., 2009). Strong learners did more pre-planning than less competent writers, and regarded an outline as a useful tool for planning and avoiding problems with the organisation of a text (Griva et al., 2009). Less able writers, on the other hand, ‘generated little content and organised it poorly’ (Griva et al., 2009, p. 141). These findings appear to indicate that the planning of text composition of young EFL learners is mainly concerned with generating ideas, activating language resources and outlining the final product. Similar results were found by Griva and Chostelidou (2013), who analysed think-aloud data and retrospective interviews. They found that the learners were mainly concerned 48 2 A Comprehensive Model of Writing Competence for Young EFL Learners with generating and organising ideas, activating background knowledge and recalling vocabulary before they started to write (Griva & Chostelidou, 2013). In terms of organising ideas, Griva and Chostelidou (2013) found that 56 % of the skilled writers organised their ideas efficiently but none of the less able writers did so (p. 3). Thus, the amount and the quality of planning done by young EFL learners appear to vary considerably. While there seem to be learners who do not plan their texts at all, others do appear to engage in planning. However, not all learners seem to do so successfully. In particular, less skilled writers might display difficulties in planning effectively, and may therefore need specific support to do so. Additionally, these studies show that in a young EFL learner context planning is not only concerned with generating and organising ideas, but also closely linked to the activation of language resources. Further research might be necessary to investigate how generating ideas and activating language resources are connected, and whether and in what way limited language resources might affect the generation and selection of ideas. Relevant research on the effectiveness of pre-writing activities on the writing quality in a young EFL learner context does not yet seem to exist. In an L1 context, however, a meta-analysis of writing instruction among elementary school learners showed that pre-writing activities which focused on generating and organising ideas had a significant positive effect on writing quality ( ES = 0.54) (Graham et al., 2012). In a study among third grade pupils, Tracy, Reid, and Graham (2009) found that explicit instruction on how to plan and draft stories resulted in higher text quality, and that this also transferred to personal narratives, a similar genre that had not been taught. Norris, Mokhtari, and Reichard (1998) researched the effectiveness of drawing as a pre-writing activity with third grade pupils. They found that pupils who drew a picture about the topic before they wrote about it composed longer and better texts than the pupils in the control group. It seems desirable for similar research on the effectiveness of different pre-writing activities to be also conducted in a young EFL learner context. 2.4.4 Formulating The process of formulating, i. e. the transformation of ideas into language, appears to be one of the key cognitive processes when writing in a foreign language, in particular with young learners and beginners. Since their language resources are strongly limited, a large amount of their cognitive capacity appears to be occupied with this process. Griva et al. (2009), in their study among young 49 2.4 Writing processes - “doing writing” EFL learners, found that weaker pupils showed considerable difficulty in paying attention to aspects such as text organisation, evaluation or revision since ‘their primary concern was to translate thoughts into words and sentences’ (p. 141). They also found that many pupils often interrupted their writing because of their limited language resources. The children themselves reported that limited vocabulary, difficulty with spelling, grammatical differences between the target language and their first language, insecurity about sentence structures and not knowing how to organise ideas restricted their writing (Griva et al., 2009, p. 143). To overcome these difficulties, the learners used strategies such as looking up words in a dictionary, translation from L1, applying L1 structures to the target language, recombining sentences, trying to find different words, adjusting the message and peer support (Griva et al., 2009). Some learners omitted difficult parts completely if they were not able to express their thoughts in the foreign language (Griva et al., 2009). Moreover, the less competent writers reported a higher level of anxiety when writing in the foreign language than more compe‐ tent writers did (Griva et al., 2009). Therefore, from a pedagogical perspective, it appears central to pay close attention to this process of formulating and to provide enough support, in particular to less skilled writers, who might otherwise be negatively affected by anxiety. Relevant findings are also reported by de Larios, Manchón, and Murphy (2006), who investigated and compared the formulation processes of L1 and EFL learners at different proficiency levels. Even though the study was conducted with learners aged 16-17, 19-20 and 23-24 with six, nine and twelve years of instruction in English and cannot therefore be directly related to younger EFL learners, the findings give relevant insights into the formulation processes of EFL learners while writing. De Larios et al. (2006) found that the learners spent twice as much time on handling formulation difficulties when writing in English as when they composed texts in their L1. The time devoted to dealing with formulation difficulties, however, was similar for all L2 learners, regardless of their language level (de Larios et al., 2006). More advanced language learners spent less time on compensating for limited language resources but more time on improving the expression of their ideas and the coherence of their texts (de Larios et al., 2006). According to de Larios et al. (2006), their data seem to indicate that ‘the lower the proficiency level of the writer, the more he or she engages in compensating for interlanguage deficits vis-à-vis ideational or textual preoccupations’ (p. 110). These findings appear to confirm again the important role of language resources in the context of foreign language writing, not only before writing but also during the formulation process. 50 2 A Comprehensive Model of Writing Competence for Young EFL Learners 2.4.5 Writing Closely related to the process of formulating appears to be the process of writing, i. e. the transcription of ideas and mentally formulated language into written text. A key aspect that may have to be considered when discussing the process of writing is the transcription tool such as handwriting, typing or also dictation and speech recognition technology, which automatically transcribes oral language into written text. Even though not much research is available from a young EFL learner context, this aspect seems to be well researched in a young L1 learner context. Berninger, Abbott, Augsburger, and Garcia (2009) found that young L1 learners in grades 2, 4 and 6 in the USA consistently wrote faster and composed longer texts by pen than by keyboard (p. 129). Connelly, Gee, and Walsh (2007) researched handwriting and keyboarding in UK primary schools. They hypothesised that handwriting would be faster than keyboarding, since the schools did not explicitly teach keyboarding skills. It was possible to confirm this hypothesis for all grades, from reception to grade 6, and only a small number of learners in grades 5 and 6 were faster at keyboarding than at writing by hand (Connelly et al., 2007). In a second study, Connelly et al. (2007) investigated the effect of handwriting and keyboarding on the text quality in grades 5 and 6. They found that the text quality was higher if the texts were written by hand, and that the quality of texts written by keyboard was ‘up to two years behind handwritten scripts in development’ (p. 479). They concluded that ‘explicit keyboarding instruction … is needed to develop keyboarding fluency and unlock the full potential of the word processor for children’s writing’ (p. 479). Read (2006) similarly found that handwriting led to higher text quality than keyboarding or the use of handwriting recognition software on a tablet computer that transformed handwritten text into digital text. The children reported that handwriting was the easiest but least fun to use, that working with handwriting recognition software on a tablet was the most difficult and keyboarding the most fun to use (Read, 2006, p. 65). It seems important, therefore, that these findings are taken into account when planning and developing writing tasks and writing instruction (see chapter 3.2.2 for information on the type of transcription used in this study). Dictation and speech recognition technology are often discussed as alterna‐ tive transcription tools for pupils with learning disabilities (see e. g. De La Paz, 1999). It is argued that ‘composing orally may allow them to circumvent transcription or text production problems (e. g., handwriting, spelling, punctu‐ ation), which in turn may allow greater focus on higher-order concerns such 51 2.4 Writing processes - “doing writing” as planning and content generation’ (De La Paz, 1999, p. 173). Since learning disabilities are not specifically targeted in this research project, this aspect will not be discussed in more detail. For a review of relevant research on the effectiveness of dictation and speech recognition software among pupils with learning disabilities, see De La Paz (1999, pp. 175-178). Besides the transcription tool, the language resources also appear to affect the transcription process, in particular in a young EFL learner context. Griva and Chostelidou (2013), in their study among young EFL learners in Greece, found that ‘the children’s writing was reportedly interrupted mid-sentence by language concerns such as spelling, grammar, word choice or struggling with putting ideas into coherent English and doubt about the meaning con‐ veyed’ (p. 4). While 81 % of the skilled writers adopted a ‘sentence-by-sentence’ approach to composing their texts, 81 % of the less skilled writers adopted a ‘word-by-word’ approach. The learners’ limited language resources, therefore, appear to considerably slow down the transcription process and, at times, even to interrupt it. 2.4.6 (Re-)Reading, evaluating, revising and editing Further key elements of ‘doing writing’ appear to be the (re-)reading, evaluating, revising and editing processes. Reading is regarded as the process of accessing information that is stored in handwritten, printed or digital form. A writer, for example, may read the task assignment, a source or sample text, the text written so far or the final product. Evaluating is defined as the process of judging the adequacy of any output produced (Hayes & Olinghouse, 2015). It may entail the rejection and confirmation of ideas, formulations, or language that has already been written down. During revision processes, ‘students “look again” at their writing holistically in order to improve such areas as organization, focus, etc.’ (Richards & Schmidt, 2010, p. 499) and when editing, they engage ‘in activities that require correction of discrete language errors in their writing, such as errors in grammar, vocabulary, sentence structure, spelling, etc.’ (Richards & Schmidt, 2010, p. 189). These different elements are closely linked to each other. A learner, for example, may read the task assignment and a sample text to evaluate whether the text he or she has written so far fulfils the task requirements and meets the expectations of that particular genre. The learner may reread the text in order to revise its content, edit linguistic errors or plan what to write next. He or she might also read a source text to get information or ideas what to write about. 52 2 A Comprehensive Model of Writing Competence for Young EFL Learners Griva et al. (2009), who investigated the composing processes and writing strategies of young EFL learners, report that most learners in their study paused during the writing process to reread what they had written and to revise certain elements. Only very few pupils wrote nonstop for a certain time without interrupting to revise or edit their texts (Griva et al., 2009). As with planning, more able writers displayed better revision skills than less able writers, who tended to revise their texts more superficially (Griva et al., 2009). Stronger writers preferred to reread and revise their texts on a paragraph level, while less able writers focused more on formal aspects (Griva et al., 2009, p. 141). McCutchen, Francis, and Kerr (1997) found similar results in an L1 context among seventh grade pupils. High-ability learners usually skimmed their texts; they ‘corrected spelling errors as they came across them and marked meaning errors for later consideration’ (McCutchen et al., 1997, p. 673). Low-ability learners displayed more uncertainty (McCutchen et al., 1997). They read each sentence individually, and ‘tried to detect problems by reading along until something sounded wrong to the ear’ (McCutchen et al., 1997, p. 673). This technique did not help the learners notice inconsistencies on the macro level in terms of content and text coherence, which they detected only rarely (McCutchen et al., 1997). Beal, Garrod, and Bonitatibus (1990), however, found that when L1 elementary school children in grades 3 and 6 were trained to ask specific questions about the content of the text when reading and revising, their ability to locate and revise problems at text level significantly improved. Koster, Tribushinina, de Jong, and van den Bergh (2015), in a meta-analysis on writing instruction among L1 learners in grades 4-6, found that evaluation (teaching the learners how to assess their texts e. g. in terms of ideas, organi‐ sation, word choice or formal conventions) and text revision had a positive but non-significant effect on writing quality ( ES = 0.43 and 0.58, respectively). Midgette et al. (2008), in a study among learners in grades 5-8, found that specific goals for text revision led to a greater improvement of text quality than the setting of unspecific goals. These findings seem to indicate, as in other areas, that there are considerable differences in the learners’ ability to (re-)read, evaluate, revise and edit their texts. The learners, and in particular less skilled writers, may benefit from specific instruction on how to evaluate, revise and edit their texts. 53 2.4 Writing processes - “doing writing” 2.5 Writing resources The resource level of the model of writing competence for young EFL learners consists of the internal and external resources the learner may draw upon during the writing process (see Figure 2.2). The internal resources are stored in the learner’s shortand long-term memory. According to Richards and Schmidt (2010), the term short-term memory ‘refers to that part of the memory where information which is received is stored for short periods of time while it is being analyzed and interpreted’ (p. 359). Long-term memory, on the other hand, is ‘that part of the memory system where information is stored more permanen‐ tly’ (Richards & Schmidt, 2010, p. 359). The resources stored in the learner’s shortand long-term memory may encompass language skills and resources in different languages, writing strategies, genre knowledge, the learner’s content knowledge and general experience, as well as any further personal resources such as motor, social or technology skills. The external resources, on the other hand, consist of any resources outside the learner which he or she might draw upon during the writing process, for example people, materials or technology. Klein and Leacock (2012) distinguish between internal representations (e. g. information in the long-term memory) and external representations (e. g. texts, graphs or tables) as important resources for writing. In addition to the internal and external representations, Klein and Leacock (2012) add collaboration as a further important resource for writing. Even though this distinction between external representations and collaborators appears to be reasonable (in partic‐ ular because of the former having a more passive and the latter a more active role during the writing process), they are subsumed under the term external resources in the model of writing competence for young EFL learners (see Figure 2.2) in order to keep the model as simple and intuitive as possible. This chapter tries to summarise what is known about the influence of different internal and external resources on the writing processes and outcomes in a young EFL learner context. It focuses on those aspects that are relevant for the design and implementation of the study and for the subsequent discussion of the results. 2.5.1 Language skills and resources As already discussed in previous chapters, the availability of language skills and resources appears to be a key element influencing the writing process and out‐ come of young EFL learners. Language skills are regarded as the learners’ ability to carry out language-related actions such as listening, speaking, reading or 54 2 A Comprehensive Model of Writing Competence for Young EFL Learners writing in order to reach a specific communicative purpose. Language resources are seen as the learners’ linguistic means such as vocabulary, pronunciation, grammar or orthography, which they use to carry out communicative tasks. Schoonen, van Gelderen, Stoel, Hulstijn, and de Glopper (2011) investigated the skills and knowledge resources EFL learners aged 13 to 17 brought to writing. They found that there was a strong relationship between the learners’ L1 and EFL writing skills and ‘that EFL writing was more strongly correlated to linguistic knowledge and linguistic fluency than L1 writing was’ (p. 32). Griva and Chostelidou (2013) found similar differences among 10to 12-year-old pupils learning Greek as their second and English as a foreign language. They reported that 6 % of the skilled writers and 75 % of the less-skilled writers relied on recalling vocabulary resources before they started to write in Greek, but that both skilled and less skilled writers did so when writing in English as a foreign language. These findings appear to support Schoonen et al. (2011)‘s findings and seem to suggest that EFL learners (and less skilled L2 writers) rely more heavily on recalling language resources than (the more skilled) L2 learners do. The majority of learners regarded the selection of vocabulary as being of paramount importance when writing in English (Griva & Chostelidou, 2013). They also reported that they thought in their first or second language and then translated their thoughts into the foreign language (Griva & Chostelidou, 2013). In addition, Griva and Chostelidou (2013) found that the learners encountered considerable language-related challenges during the writing process. The pupils, for example, reported that they were sometimes ‘interrupted mid-sentence by language con‐ cerns such as spelling, grammar, word choice or struggling with putting ideas into coherent English and doubt about the meaning conveyed’ (p. 4). In order to overcome such language-related difficulties, the learners applied different strategies (Griva & Chostelidou, 2013). Stronger learners tended to search for different words or adjust the message, while less able writers preferred to ask for help (Griva & Chostelidou, 2013). Some learners switched to writing in their L1 or L2, avoided certain expressions, or even stopped writing (Griva & Chostelidou, 2013). These findings show that language skills and resources appear to play a prominent role when young EFL learners compose texts. It thus seemed central to include them as an explicit aspect in the model of writing competence for young EFL learners (see Figure 2.2) and to investigate whether similar findings could be found in the context of this study (see chapter 4.2.2 about the pupils’ perception of EFL writing at primary school). It also appears central that limited foreign language skills and resources are not primarily regarded as a disturbing factor that prevents the learners from displaying their actual writing competence. Against the background of 55 2.5 Writing resources competency-oriented language learning, it seems important that the children’s language skills and resources, limited as they may be, should be positively recognised as an integral part and key component of their writing competence. Chapter 2.8 discusses in detail what kind of language skills and resources young EFL learners display in their writings. 2.5.2 Writing strategies and genre knowledge After discussing the learners’ language skills and resources, the chapter now turns to discuss their knowledge about writing strategies and genres. According to Hayes and Olinghouse (2015), writing strategies refer to a writer’s ‘knowledge about how to go about producing texts’ (p. 483). This knowledge may relate to any of the different levels and elements of the model of writing competence for young EFL learners (see Figure 2.2). Strategies related to the target level of writing, for example, may include the learner’s knowledge of how to consider the reader’s interests or how to assess whether the text fulfils its purpose. Strategies related to the performance level may encompass their knowledge of how to plan or revise their texts, or how to obtain feedback. Strategies related to the resource level of writing might include, for example, the learners’ knowledge of how to use the different types of materials and technology that could be available during the writing process (e. g. dictionaries, text models, books or the internet), or how to ask the teacher or peers for support. Strategies related to the product level may encompass their knowledge of how to assess whether and how well their texts implement important genre-specific aspects, whether the content is structured in a logical way or what needs to be improved in terms of language. Strategies related to the learners’ executive and monitoring function may include their knowledge of how to focus their attention on relevant aspects or also metacognitive strategies such as monitoring the efficiency of different strategies. Strategies related to personal and external determinants might, for example, entail the learners’ knowledge of how to regain concentration after a distraction, or of how to create a suitable writing environment, e. g. by searching for a quiet place in which to write. According to Griva et al. (2009), young EFL learners aged 12 in Greece used strategies such as planning, brainstorming, collecting vocabulary, creating outlines, reading a model text, asking for help, clarification and correction, collaborating with peers, working alone, using resources, rereading to revise the text, paying attention, focusing on specific aspects, self-encouragement and rewarding themselves. With regard to language-related difficulties, the learners applied strategies such as using expressions they had recently learnt 56 2 A Comprehensive Model of Writing Competence for Young EFL Learners in class, looking up words in a dictionary, translating from L1, applying L1 patterns and rules to the foreign language, simplifying the wording, trying to find different words, avoiding difficult parts, recombining sentences and adjusting the meaning (Griva et al., 2009). Chapters 4.2.2 and 4.2.1 report on what strategies were used by the learners in this study, and on what strategies the teachers used or promoted in their classes. However, Griva et al. (2009) also found that poor writers used writing strategies less often than good writers did, and that the quality of how they used the strategies was lower compared to their more able peers (p. 141). This finding seems to be in line with Boscolo (2008), who argues that ‘what distinguishes competent from less competent and struggling writers seems to be a repertoire of strategies that the former can flexibly use when planning, composing, and revising their texts’ (p. 368). The aspect of genres as a framework for specifying the target level of a writing task was discussed in chapter 2.3.2. This chapter discusses genre knowledge as a writing resource. According to Hallet (2016), genre knowledge is acquired through socialisation and repeated encounters with different types of discourse and text, and consists of different dimensions: knowledge of shared communicative purposes, knowledge of text conventions, knowledge of content and knowledge of the cultural context (pp. 49-50). While in the initial stages of foreign language learning at primary school generic learning may be an intuitive encounter and mainly concern habit formation, the development of socially and culturally conventionalised schemata and pre-lingual standardisa‐ tion, later stages of generic learning may also encompass the more explicit use of genre-related terms and conscious categorisation (Hallet, 2016, p. 47). According to Hayes and Olinghouse (2015), genre knowledge gives guidance on how to write texts of different genres and provides support for planning and revising texts. Genres may serve as models for language production and social interaction (Hallet, 2016, p. 104). Furthermore, they may be used as instructional scaffolding, provide opportunities for experiencing successful communication and, at later stages, give room for individual creativity (Hallet, 2016, p. 93). Research on young EFL learners’ genre knowledge seems rare. Donovan and Smolkin (2002) analysed children’s genre knowledge in an English L1 context in the USA . At kindergarten level, they found that some children could already express some basic genre knowledge, e. g. that information books are ‘true’ and stories ‘not real’, or that a text beginning with ‘once upon a time’ is a story (Donovan & Smolkin, 2002, p. 456), while others did not know these differences or gave very simple answers, e. g. that ‘a story is a story’ (p. 452). According to Donovan and Smolkin (2002), the learners in upper elementary 57 2.5 Writing resources school displayed a more detailed genre knowledge than lower elementary school learners. Some pupils, for example, showed an awareness of genre-specific linguistic characteristics of a text, were able to differentiate between real and imaginary stories, mentioned that stories contained actions and a plot, or that information books ‘served to explain things to people’ (Donovan & Smolkin, 2002, p. 457). Whether young EFL learners have a similar genre knowledge can only be assumed. Clearly more research appears necessary to determine whether the genre knowledge of young learners in a first language differs from that in a foreign language, and how well young EFL learners are able to use this knowledge for writing. The study presented here focuses on this second aspect, namely to what extent and how the learners are able to apply genre-specific aspects in their writings. Chapter 4.1.3 gives a descriptive account of the different qualities of the young EFL learners’ texts, including the genre-specific elements of addressing the reader in correspondence and the communicative effect and genre-specific structure of narrative texts. In addition, chapters 4.1.5 and 4.1.6 specifically analyse the latter two aspects in more detail (communica‐ tive effect and coherence in narrative texts), trying to determine their specific characteristics and differences between the language levels. 2.5.3 Content knowledge and experience The term content knowledge and experience is used to refer to the conceptual and experiential knowledge the learner may draw upon during the writing process. Besides knowledge of the topic, this may encompass further types of knowledge such as knowledge of the reader, the remembrance of personal experiences and events, or knowledge of other people’s experiences. Compared to adult or more experienced writers, young learners may gener‐ ally have less content knowledge and experience, even though they may exceed adults in certain areas of knowledge with which they are highly familiar. The amount of available content knowledge may affect the number and quality of ideas that can be generated and, eventually, have an impact on the quality of writing. Olinghouse, Graham, and Gillespie (2015), in a study on 5th-grade L1 learners, found that topic knowledge was a predictor of the quality of narrative, argumentative and informational texts. Similar findings are reported by DeGroff (1987) and Mosenthal, Conley, Colella, and Davidson-Mosenthal (1985) among 4 th -grade L1 learners and by McCutchen (1986) among 4 th - , 6 th and 8 th -grade pupils. 58 2 A Comprehensive Model of Writing Competence for Young EFL Learners No research findings appear to be available on topic knowledge in a young EFL learner context. It might be desirable, therefore, for further research to be conducted to investigate whether the findings just mentioned also apply in this context, and what influence the learners’ language resources may have on how well the learners can make use of their topic knowledge when writing in a foreign language. Despite this lack of research, it appears central to consider the learners’ content knowledge and experience when selecting and developing writing tasks. If the learners are not familiar with the topic, or if it is too far beyond their experience, they may not be able to display their full writing skills, which would affect the validity of the results. Moreover, if topics are selected on which the learners have largely diverging levels of prior knowledge, this may similarly lead to biased results. For information on how the topics for the writing tasks were selected in this study, see chapter 3.2.2. 2.5.4 External resources: people and materials In addition to internal resources, the learners may also access external resources while writing. Such external resources may encompass any resources outside the learner that are available to them during the writing process, for example people, materials or technology. The first two, people and materials, are relevant for this study, and thus discussed in this chapter. Firstly, the young EFL learners may draw upon the resource people, such as peers, teachers or family members. These people may, for example, co-author the text, ask or answer questions, provide support and opportunity for discussion, or give feedback. According to Zhang and Patel (2006), collaboration can be beneficial because additional resources may be available, the task and memory load can be shared, and errors be mutually corrected. Collaboration, on the other hand, may also have disadvantages, for example because more time for discussion might be needed, or because the members of a group might not share their knowledge or prefer different ways of working (Zhang & Patel, 2006). Griva et al. (2009) found that the majority of the young EFL learners participating in their study regarded collaboration with peers as a helpful means to overcome difficulties. Most learners, and in particular less able writers, appreciated working with someone else, while some strong learners indicated that they preferred to work alone (Griva et al., 2009, pp. 144-145). A greater number of studies about collaboration in writing is available in a young L1 learner context. Several studies showed that collaboration during the writing process can have a positive effect not only on the writing quality but also on 59 2.5 Writing resources the learners’ motivation to write (e. g. Graham et al., 2012; Koster et al., 2015; Li, Chu, & Ki, 2014; Yarrow & Topping, 2001). In addition to the collaboration with other people, various materials such as task and source materials or dictionaries could be drawn upon. According to Zhang (1997), such materials can have different functions. They may, for example, serve as memory aids, allow the sharing of information, provide access to resources that might not be available in the internal resources, help to organise thoughts or even change the nature of a task (Zhang, 1997). Griva et al. (2009) and Griva and Chostelidou (2013) found differing results regarding the use of dictionaries among young EFL learners. Griva et al. (2009) report that many learners in their study regarded looking up words in a dictionary as the most useful and most frequently used tool to check the meaning or spelling of words. Griva and Chostelidou (2013), on the other hand, found that ‘75 % of the less-skilled writers and 56.3 % of the skilled writers did not use a dictionary’ (p. 5). The reason for this difference cannot be fully clarified. The frequency with which young EFL learners use dictionaries while writing might depend on the availability of dictionaries in the classroom, or also on how much emphasis is placed on their use by the teacher, or on how frequently looking up words in a dictionary is practised in class. In terms of task and source materials, Griva et al. (2009) found that the learners ‘showed a major preference (35 % sometimes and 30 % frequently) for having the opportunity to read a text as a model for the particular function’ (p. 136). Receiving a list with useful words was slightly less popular, with about 56 % of the learners showing some preference for it (Griva et al., 2009, p. 136). While the use of external resources appears to be a relevant component of learning to write in a foreign language, their role when testing and assessing the learners’ writing competence seems to require careful consideration. In class‐ room assessment, the use of external resources as instructional scaffolding (e. g. the use of text models, dictionaries or peer collaboration) appears reasonable, since their use might closely reflect classroom instruction and, therefore, be part of the competences that are to be assessed. In the context of large-scale tests and research, the decision on whether external resources should be included when testing the learners’ writing competence may largely depend on the construct definition of writing, the purpose of the study and the measures taken to ensure validity and reliability. For details about how this question was addressed in this study, see chapter 3.2.2 (external resources and transcription tool). 60 2 A Comprehensive Model of Writing Competence for Young EFL Learners 2.6 The learner’s executive and monitoring function In a young EFL learner context, where the pupils have limited language re‐ sources and a comparably small writing expertise, and where various processes are not as fluent as with older and more experienced learners, it seems partic‐ ularly important to examine the role of the learner’s executive and monitoring function in order to understand the specific characteristics and challenges of EFL writing at primary school. The learner’s executive and monitoring function is regarded as embracing the performance level of writing and all mental and physical activities that contribute to the writing process (see Figure 2.2). It comprises the writer’s ability to access, process and combine different types of information, to focus and maintain attention, take action, and monitor and evaluate progress and various influencing factors. The amount of information that can be processed, the number of aspects that can be attended to and the quality of the executive and monitoring processes may depend, inter alia, on how much cognitive capacity is currently available, on how much working memory is required for the different processes and on how experienced the writer is. The working memory of a novice writer, for example, who is not yet fluent in handwriting and does not have much writing expertise, may mainly be devoted to the message the child wants to convey, to remembering the shapes of letters and to controlling fine motor skills. A young EFL learner at a slightly older age may need most of their cognitive capacity to find suitable words, expressions and sentences in the foreign language with which they can express their ideas. As already discussed in earlier chapters, such observations have been made by Griva et al. (2009). The young EFL learners in their study reported on what difficulties they encountered during the writing process. While they did not mention handwriting to be a problem (possibly because they were in grade 6 and largely fluent in handwriting), they declared that aspects such as finding appropriate words, structuring sentences, adequately developing the content, formulating and organising ideas and difficulties with grammatical structures and spelling constrained their writing performance (Griva et al., 2009). Griva and Chostelidou (2013) compared the composing processes of learners aged 10-12 when they composed texts in Greek as a second language, and in English as a foreign language. They found that the learners’ use of strategies and their ability to monitor their writing were lower in English than in Greek, and concluded that this might be due to their lower language proficiency in English (Griva & Chostelidou, 2013, p. 8). Grotjahn (2017) similarly argues that limited foreign language resources might not only affect the quality of a written product, but 61 2.6 The learner’s executive and monitoring function also lead to considerably higher demands on the writer’s cognitive capacity. As a consequence, less processing capacity might be available, and the transfer of L1 writing expertise (such as genre knowledge or writing strategies) onto L2 writing might be impeded or even made impossible (Grotjahn, 2017, p. 98). It therefore seems essential for EFL teachers to be aware of the different aspects that occupy the learners’ cognitive capacity while writing. Knowing about these aspects might enable the teachers to purposefully plan writing tasks and procedures that correspond with the learners’ language proficiency, and to provide an instructional scaffolding that enables the learners to be successful and develop a positive attitude towards writing in the foreign language. This appears to be particularly important, since Griva et al. (2009) observed that in particular less able writers may sometimes be overwhelmed and ‘discouraged by the demands of writing‘ (p. 141). Such knowledge may also help the teachers to set appropriate and manageable product and process goals (see chapter 2.3.1) that help the learners focus their attention on relevant aspects of their writing, for example when drafting or revising their texts. In order to provide the teachers with such knowledge, the study investigated the learners’ perception of EFL writing, including what they like and dislike about writing in English and the challenges they encountered during the writing process (see chapter 4.2.2). 2.7 Individual and external determinants Various individual and external determinants (see Figure 2.2) appear to be closely related to the learner’s executive and monitoring function. Individual determinants are regarded as personal factors that influence the learner’s performance either positively or negatively, such as motivation or emotions (see e. g. Bai & Guo, 2019; Tsiriotakis, Vassilaki, Spantidakis, & Stavrou, 2017). External determinants, on the other hand, are factors relating to the learners’ environment that affect their performance, such as noise, temperature or air quality (see e. g. Coley, Greeves, & Saxby, 2007; Sörqvist, Nöstl, & Halin, 2012; Wargocki, Wyon, Matysiak, & Irgens, 2005). This research study focuses on two of the individual determinants which appear to be of particular relevance in a young EFL learner context, but do not seem to have been well researched up to now, namely young EFL learners’ motivation for and their self-efficacy in writing in English. Since it appears that only a small amount of research from a young EFL learner context is available, research from a young L1 learner context is reported instead. 62 2 A Comprehensive Model of Writing Competence for Young EFL Learners 2.7.1 Young EFL learners’ motivation to write As discussed in chapters 2.3.3 and 2.6, a central aspect of teaching foreign languages to young learners is to give them opportunities to be successful in using the new language for communicative purposes. Since they still have many years of learning ahead of them, it appears particularly important to help them develop and maintain a positive attitude towards learning in general, and also towards EFL writing. Boscolo (2009) conceptualises children’s motivation to write as their ‘perception of the meaningfulness of school writing and sense of competence’ (p. 302). Bruning and Horn (2000), drawing on relevant research about writing and motivation, summarise four key factors that appear to have a positive effect on writing motivation: nurturing functional beliefs about writing, fostering student engagement through authentic writing goals and contexts, providing a supportive context for writing and creating a positive emotional environment (pp. 27-28). Nolen (2007) researched children’s motivation to write in a quali‐ tative study in an L1 elementary school context in the USA and found similar results. The children mentioned mastery, interest, enjoyment, creativity, choice and self-expression as the sources of their motivation for writing (Nolen, 2007, p. 231). In a class where the children wrote their texts for a real audience (family, friends, teacher) and where projects gave the learners a choice, the children referred to writing with ‘language of privilege and positive emotion’ (Nolen, 2007, p. 259). In another class, where the writing tasks were more controlled and the teacher acted as the main audience, writing ‘was primarily seen as a school task’ (Nolen, 2007, pp. 258-259) and the children tended to use language of obligation when talking about writing. Since little is known about young EFL learners’ motivation to write, this study investigated their perception of EFL writing and possible sources of their motivation to write (see chapters 4.2.2 and 5.2.6). 2.7.2 Young EFL learners’ self-efficacy Closely linked to the learners’ motivation to write appears to be their self-effi‐ cacy, i. e. a person’s perception of his or her ability to successfully complete a future task (Troia, Harbaugh, Shankland, Wolbers, & Lawrence, 2013). Boscolo (2009) regards self-efficacy as a key element of children’s motivation to write. Various studies show that self-efficacy is a strong predictor of primary school learners’ L1 writing competence (e. g. Bulut, 2017; Pajares & Valiante, 1997), and Troia et al. (2013), in a study among learners in grades 4-10 in the USA , also 63 2.7 Individual and external determinants found that motivation, including the learners’ self-efficacy, had a significant positive effect on writing quality. While young learners’ self-efficacy in writing seems to be well researched in an L1 context (see e. g. Klassen, 2002; Pajares, 2003, p. 152 for reviews), not much research appears to be available from a young EFL learner context. Bai and Guo (2019) researched 4 th grade EFL learners’ self-efficacy in relation to self-regulated strategy use when writing in English and found a high and statistically significant correlation between the two aspects. With regard to writing proficiency, however, no studies from a young EFL learner context could be found. Thus, this study tried to investigate to what extent young EFL learners’ self-efficacy is a predictor of their EFL writing competence (see chapter 3.4.4 for information about the scales used for measuring the learners’ self-efficacy and chapter 4.3.1 for the results). 2.8 The writing product In the context of competency-based language teaching, writing competence is usually measured with tasks that simulate or represent real-life activities (Grotjahn, 2017). From the learners’ performance in these tasks, conclusions are drawn about their competence (Grotjahn, 2017). In contrast to indirect measures such as multiple-choice items, or tests of grammar and lexis, this type of measurement not only assesses the existence of various resources for writing but the extent to which the learner is able to use these resources in communicative situations (Grotjahn, 2017, pp. 72-73). It assesses writing in all its complexity, as shown in the model of writing competence in chapter 2.2.3. This view is in accordance with pragmatic text linguistics, which tries to describe and explain the conditions of sociolinguistic communication between members of a particular discourse community (Brinker, Cölfen, & Pappert, 2018, p. 15). Pragmatic text linguistics regards a text as a complex, communicative act with which the author tries to establish a communicative relationship with the reader (Brinker et al., 2018). A central aspect of pragmatic text linguistics is the communicative function of a text, which shapes both the selection of linguistic elements and the development of the content (Brinker et al., 2018). According to Brinker et al. (2018), pragmatic text linguistics and structural linguistics, which investigates the language system, should be regarded as complementary approaches rather than alternatives. They argue that an adequate linguistic text analysis should take both approaches into account and describe a text as a linguistic and communicative entity (Brinker et al., 2018, p. 17). 64 2 A Comprehensive Model of Writing Competence for Young EFL Learners This, in turn, appears to be in line with the definition of communicative competence as described in the CEFR (Council of Europe, 2001). Communicative competence is seen as a combination of linguistic, sociolinguistic and pragmatic competences (Council of Europe, 2001, p. 13). Linguistic competence refers to the ‘knowledge of, and ability to use, the formal resources from which well-formed, meaningful messages may be assembled and formulated’ (Council of Europe, 2001, p. 109) and may include aspects such as vocabulary, grammar, orthography and punctuation. Sociolinguistic competence is seen as ‘the knowledge and skills required to deal with the social dimension of language use’ (Council of Europe, 2001, p. 118) and may include aspects such as addressing the reader, register (e. g. the use of formal or informal language) and politeness. Pragmatic competence is concerned ‘with the functional use of linguistic resources’ (Council of Europe, 2001, p. 13) and refers to the knowledge of and ability to apply principles of text organisation (e. g. coherence and cohesion), text design (e. g. genre knowledge) and the ability to pursue particular text functions (Council of Europe, 2001, p. 123). Therefore, it appears to be central that the analysis of text quality is not only based on the linguistic characteristics of a text, but also considers sociolinguistic and pragmatic aspects. This chapter tries to summarise what is known from research about the qualities of young EFL learners’ texts and, subsequently, discusses the findings in relation to the CEFR descriptors relevant for young learners aged 11-15 as described in Szabo (2018). These considerations form the basis for the operationalisation of text quality in this study (see chapter 3.2.1). First, pragmatic aspects will be discussed, followed by sociolinguistic and linguistic aspects. 2.8.1 Pragmatic text qualities Communicative effect One of the key elements of pragmatic text quality is the extent to which a text fulfils its communicative function. The text function is closely linked to the text genre. In a young learner context, written products may have, for example, a descriptive, instructive, explanatory, persuasive or entertaining function (Rietdijk, Janssen, Van Weijen, Van den Bergh, & Rijlaarsdam, 2017). Therefore, in order to assess this quality, questions need to be asked such as: How informative is the description? How entertaining is the story? Or how persuasive is the argument? 65 2.8 The writing product Many studies acknowledge the importance of the communicative function of a text and apply rating criteria such as communicative effect (Harsch, Schröder, & Neumann, 2008; Konsortium HarmoS Fremdsprachen, 2009), communicative effectiveness (Rietdijk et al., 2017) or communicative quality (Schoonen et al., 2011). Qualitative descriptions of how the learners achieve this communicative effect, however, are rare and do not seem to exist in a young EFL learner context. It appears important, therefore, that research be conducted which not only measures the communicative quality of the learners’ texts in a quantitative way but also analyses the specific implementation of this dimension. Thus, a small qualitative analysis (n = 25) was carried out in this study in order to investigate how young EFL learners at different language levels create a communicative effect in narrative texts (see chapters 3.6.1 and 4.1.5). Coherence and cohesion Two further elements that are categorised among the pragmatic competences in the CEFR are coherence and cohesion (Council of Europe, 2001, p. 123). According to de Beaugrande and Dressler (2013), cohesion ‘concerns the ways in which the components of the S U R F A C E T E XT , i. e. the actual words we hear or see, are mutually connected within a sequence’ (p. 3, emphasis in the original). Coherence, on the other hand, is defined as ‘the ways in which the components of the T E XT UAL W O R L D , i. e. the configuration of C O N C E P T S and R E LATI O N S which underlie the surface text, are mutually accessible and relevant’ (de Beaugrande & Dressler, 2013, p. 4, emphasis in original). In other words, cohesion refers to the linguistic connectivity of a text while coherence is concerned with the logical organisation and structure of the content. In terms of descriptions of text coherence, not much is known about texts from a young EFL learner context. Butler and Zeng (2014) researched the coherence in oral narratives of young L1 and EFL learners in grades four, six and eight. They used a model of story complexity originally presented by Stein and Policastro (1984) and found that the majority of EFL learners in grade four told stories that had no real structure, were mainly descriptive or consisted of a series of actions (Butler & Zeng, 2014, pp. 80-86). The stories of the learners in grades six and eight showed more variability (Butler & Zeng, 2014, p. 85). Some of these stories additionally included causal relations and goal-based episodes but many were still lacking an obstacle or ending (Butler & Zeng, 2014). However, there were also stories that contained all elements of Stein and Policastro’s model, including an obstacle and an ending (Butler & Zeng, 2014, pp. 86-87). In comparison to the stories told in the learners’ L1 (Chinese), the scores for coherence of the stories told in English were generally lower at all grades 66 2 A Comprehensive Model of Writing Competence for Young EFL Learners (Butler & Zeng, 2014, p. 85). No descriptive research on coherence in young EFL learners’ written narratives appears to exist. There is clearly a need for more research with regard to this aspect. Therefore, a small qualitative analysis (n = 25) of the coherence in the young EFL learners’ narrative texts was conducted in this study. Chapter 3.6.1 provides some details of methodological considerations and chapter 4.1.6 presents the results. In terms of cohesion, Yasuda (2019) analysed texts written by young EFL learners in Japan using the categories of cohesion as suggested by Halliday and Hasan (1976). Pupils who learned English in a CLIL setting (i. e. content and language integrated learning, where not only English but also other school subjects such as maths and science are taught in the foreign language) used a significantly wider range of cohesive devices than learners who studied English in regular EFL classes (Yasuda, 2019). Among the most frequently used cohesive devices were additive conjunctions (e. g. and), repetitions of the same word, causal conjunctions (e. g. because) and personal pronouns (e. g. he, she, they, it) (Yasuda, 2019). Further cohesive devices found in the texts were, for example, temporal conjunctions (e. g. first), hyponymy (the use of specific and general words from the same semantic field, e. g. banana/ fruit), adversative conjunctions (e. g. but), demonstratives (e. g. this) and comparatives (e. g. more) (Yasuda, 2019). Hasselgreen and Sundet (2017), in a corpus analysis of young Norwegian EFL learners’ texts, found considerable differences regarding the use of conjunctions between 12 / 13 and 15 / 16 year-old learners. Several conjunctions and adverbial conjuncts (however, since, even if, although, even though and therefore), which were sometimes used by the older learners, almost never appeared in younger learners’ texts (Hasselgreen & Sundet, 2017, p. 210). An exception was the conjunction because, which was used frequently by both groups (Hasselgreen & Sundet, 2017). For information on how cohesion was measured in this project, see chapter 3.2.1. For examples of how cohesive devices and reference words were used by the young EFL learners in this project, see the six sample profiles in chapter 4.1.3. 2.8.2 Sociolinguistic text qualities As discussed before, sociolinguistic competence ‘is concerned with the knowl‐ edge and skills required to deal with the social dimension of language use’ (Council of Europe, 2001, p. 118). The question arises as to whether and to what extent young EFL learners are capable of considering this social aspect when writing in a foreign language. While reader orientation appears to be central in every genre, it seems of particular importance in correspondence. Yasuda (2019) 67 2.8 The writing product and Lindgren and Stevenson (2013) investigated the language used by young EFL learners to establish a relationship between themselves and the readers of their letters. The learners in Yasuda’s study (2019) were young Japanese EFL learners in grade 5. The children wrote personal letters to introduce themselves to a host family with whom they would stay in the summer (Yasuda, 2019). All learners included the elements salutation (e. g. Dear Johnson family), frame (e. g. Hello. My name is … Thank you for your letter.), provided the reader with information and asked them questions (Yasuda, 2019). Further elements that were used by the majority of learners were leave-taking (e. g. Hope to see you soon.) and signing-off (e. g. bye bye). Less frequently used aspects were making requests (e. g. Please come to Japan.), making an offer (e. g. I will show you …) and expressing gratitude (e. g. Thank you very much for letting me stay …) (Yasuda, 2019). Moreover, Yasuda (2019) found that the children used drawings, for example to reinforce the expression of feelings or to illustrate objects the reader might not be familiar with. In terms of the expression of attitude, Yasuda (2019) found that the pupils more frequently expressed their feelings and preferences (e. g. I like … I want to go to …) than they evaluated human behaviour, objects or ideas (e. g. Mr Morley is very kind and cool.). In terms of graduation, the children used a neutral tone (e. g. I like … I don’t like …) more frequently than emphasis (e. g. I’m very excited.) and hedges (e. g. a little bit sad) (Yasuda, 2019). Lindgren and Stevenson (2013) conducted a similar study with young Swedish EFL learners in grade 5 and found similar results. The pupils likewise used non-linguistic resources such as emoticons and expressed feelings and prefer‐ ences more often than they evaluated behaviour, objects or ideas (Lindgren & Stevenson, 2013). However, they used the interaction elements frame and leave-taking less frequently than salutation, asking questions and sign-off (Lindgren & Stevenson, 2013). Furthermore, in terms of graduation, they used emphasis more often than neutral language, and hedges only rarely (Lindgren & Stevenson, 2013, p. 399). Thus, it can be concluded that young EFL learners are indeed able to establish basic social contact through writing. There seems to be a certain familiarity with genre-specific elements such as salutation, frame, asking and answering questions, or leave-taking and signing-off, and the learners mainly appear to use neutral language. Formal or polite language such as linguistic hedging do not yet seem to be used. This could be due to the fact that the pupils learn the language as a foreign language, and may therefore only rarely encounter this type of language in their social environment outside the classroom, or it could be due to the fact that more formal and polite language often requires more complex 68 2 A Comprehensive Model of Writing Competence for Young EFL Learners language (e. g. conditionals or modal verbs), which are forms young EFL learners might not yet have acquired. Because of their importance for establishing social contacts, the use of genre-specific elements in correspondence was included in the rating scales developed for this project (see chapter 3.2.1). 2.8.3 Linguistic text qualities Vocabulary Lexical diversity is generally considered a strong predictor of writing compe‐ tence. However, while some studies reveal a high correlation between vocabu‐ lary range and writing competence (e. g. Engber, 1995; Stæhr, 2008), there appear to be certain factors that might influence these outcomes such as the learners’ L1 ( Jarvis, 2002; Yu, 2009) or the genre and task (Olinghouse & Wilson, 2013; Wang, 2014). In a young EFL learner context, Llach and Gómez (2007) analysed the vocabulary used by young Spanish EFL learners in written composition. The learners were 9-10 years old, in grade 4 and had started to learn English at nursery school (Llach & Gómez, 2007). Llach and Gómez (2007) found that the children mainly used everyday vocabulary from familiar semantic fields (even though this may have also been influenced by the selection of the task). They used personal and possessive pronouns as well as simple verbs such as have, be, live, like, love, can and go (Llach & Gómez, 2007). Llach and Gómez (2007) also found that the children used nouns more frequently than verbs, and that verbs were often omitted. Llach and Gómez (2007) also investigated the different types of lexical errors in the learners’ texts and found that misspellings, omissions, borrowings and substitutions were the four main types of lexical errors. Since misspellings are concerned with orthographic control, this type of error will be discussed below (see Orthography). According to Llach and Gómez (2007) the learners mainly omitted verbs (e. g. to be and to have) and pronouns (e. g. it and I). The omission of pronouns appears to be related to the learners’ L1 (Spanish), where the use of subject pronouns is not mandatory (Llach & Gómez, 2007). Furthermore, Llach and Gómez (2007) observed that the learners borrowed Spanish words and directly included them in the English text without any adaptations, presumably because they did not know a corresponding English expression. Besides borrowing words, the children also used L1 substitutes, e. g. they used Spanish words and adapted them so that they resembled English words (Llach & Gómez, 2007). Lasagabaster and Doiz (2003) found that 11-12 69 2.8 The writing product year-old EFL learners borrowed more words from their L1 than older learners (aged 15-16 and 17-18) who had experienced a similar amount of L2 exposure time. Lindgren and Stevenson (2013), in a study among young Swedish EFL learners, additionally found that the learners translated L1 words into the English language (e. g. Min älsklingsmusik är … => My love music is …) or used English substitutes (e. g. My favourite eat … or My hobbies are gaming and eating.) (p. 401). Grammar: Syntax and morphology According to the CEFR , ‘the grammar of a language may be seen as the set of principles governing the assembly of elements into meaningful labelled and bracketed strings (sentences)’ (Council of Europe, 2001, pp. 112-113). The study of grammar is concerned with the two aspects morphology (internal structure of words) and syntax (combination of words into sentences) (Meyer, 2008, p. 11). Lasagabaster and Doiz (2003) found that the texts of 11-12 year-old EFL learners consisted of shorter sentences and were of less syntactical and dis‐ coursal complexity than those of learners aged 15-16 and 17-18 with a similar amount of experience in the target language. Grammatical correctness, however, did not always increase with age (Lasagabaster & Doiz, 2003). Longer and more complex texts accounted for certain types of grammatical errors that did not appear in younger learners’ texts, since such advanced structures were simply not used at a younger age, or to a smaller extent (Lasagabaster & Doiz, 2003). 11-12 year-old learners, for example, almost exclusively used the two tenses present simple and present continuous while older learners used a wider range of verb tenses (Lasagabaster & Doiz, 2003). Hasselgreen and Sundet (2017) made a similar observation. In a brief analysis of texts from the C O R Y L corpus (corpus of young learner language), they found that learners at the CEFR level A1 tended to mainly use the verb ‘was’ to express the past tense. Learners at the CEFR level A2 used, albeit with many errors, a greater variety of past tense forms (Hasselgreen & Sundet, 2017, p. 207). These variable differences between complexity and correctness of grammatical structures in the learners’ texts proved to be a considerable challenge for the development of reliable rating scales in this study. Chapter 3.2.1 describes how this challenge was dealt with. Raaen and Guldal (2012) compared the syntactical complexity of texts written by young Norwegian EFL learners in grades 7 (age 12-13) and 10 (age 15-16). They found that grade-7 learners most frequently used simple sentences (44 %), followed by complex sentences with at least one main and one dependent clause (29 %) and compound sentences with two or more main clauses (21 %). This 70 2 A Comprehensive Model of Writing Competence for Young EFL Learners proportion changed to 53 % complex sentences, 28 % simple sentences and 18 % compound sentences in grade 10 (Raaen & Guldal, 2012, p. 110). Lindgren and Stevenson (2013) analysed texts written by young Swedish EFL learners in grade 5. With regard to grammatical structures, they found that the pupils used various resources or strategies to overcome the obstacles caused by their limited L2 proficiency. The children, for example, translated grammatical structures from their L1 into the English language (e. g. Vad har du för hobby? => What have you four hobby? ), used grammatical substitution such as i am not like school so muth (I don’t like school so much) or simplified the message (e. g. I love school but it is so urly instead of I go to X school and I think that you learn a lot there but I don’t like to get up in the morning). In the study by Griva et al. (2009) among young Greek EFL learners, the pupils reported the use of similar strategies: translation from L1; applying L1 rules to the foreign language; simplification of the word usage when encountering a difficult sentence, phrase or word; skipping difficult parts; recombining phrases into new sentences or adjusting the meaning (p. 140). As can be seen in chapter 4.1.3, the use of such strategies could also be observed in the young EFL learners’ texts in this study. Orthography Orthography is concerned with the norms of how oral language or phonemes (smallest units of spoken language) are represented in written form (or gra‐ phemes, the smallest units of written language) (Meyer, 2008, p. 154). According to Meyer (2008), the English orthography is considered particularly difficult ‘be‐ cause the correspondences between graphemes and phonemes in this spelling system seem extremely inconsistent and unreliable’ (p. 156). While there are certain regularities, some phonemes can be represented by various graphemes, some graphemes can represent several phonemes, and there are also silent letters, which are written but not pronounced (Meyer, 2008, p. 156). Llach and Gómez (2007) analysed orthographic errors in texts written by young Spanish EFL learners. The analysis revealed that the learners often tended to write words in a way that reflected oral pronunciation (Llach & Gómez, 2007). This was equally found for nouns, verbs, adjectives and adverbs (Llach & Gómez, 2007). Vogt and Bader (2017) researched various aspects of orthographic control among EFL learners in grades 3 to 6 in Switzerland and among L1 learners in the same grades in Great Britain. In their analysis of the EFL learners’ orthography, they found that one-syllable words (e. g. play, vet), frequently used words (e. g. and, do, the) and words that have the same phoneme-grapheme relationship in English and in the German language (e. g. is, in, past) were mostly written without errors. Error-prone, on the other hand, were words with sounds that 71 2.8 The writing product do not exist in the German language or have a different phoneme-grapheme relationship, words with silent letters and complex grapheme constellations, and multi-syllable words (Vogt & Bader, 2017). Among the most error-prone aspects were words starting with ‘wh’ (e. g. what, when, where and in particular who and which), capitalisation, the grapheme ‘th’ and words with apostrophes (Vogt & Bader, 2017). Vogt and Bader (2017) observed that these errors were generally more frequent at lower grades than at higher grades (with some exceptions such as the capitalisation of days and months or the use of apostrophes). Similar findings were reported by Raaen and Guldal (2012), who analysed orthography in texts written by Norwegian EFL learners in grades 7 (end of primary school, age 12-13) and 10 (end of secondary school, age 15-16). They similarly found that the learners in grade 10 made fewer orthographic errors than the learners in grade 7, but that genitive forms with apostrophes remained difficult even in grade 10 (Raaen & Guldal, 2012). Lasagabaster and Doiz (2003) compared the orthography of three groups of Spanish and Basque EFL learners who differed in terms of age (11-12 years, 15-16 years and 17-18 years) but who had all been exposed to the target language to a similar extent. They also found that the youngest group made more spelling mistakes than the older learners did and that their spelling was highly influenced by L1 pronunciation and spelling. The influence of the learners’ L1 was found to be much smaller with the older learners (Lasagabaster & Doiz, 2003). Punctuation Punctuation refers to ‘the use of graphic marks such as commas, semicolons, dashes and periods to clarify meaning in written sentences or to represent spoken sentences in writing’ (Richards & Schmidt, 2010, p. 474). Research on the use of punctuation in a young learner context is rare, and even more so in a young EFL learner context. Therefore, results from studies among young L1 learners are presented here. Ferreiro and Pontecorvo (1999) researched punctuation in narrative texts written by L1 primary school learners from Spain and Italy. They found that the most frequently used punctuation marks were full stops and commas, followed by questions marks, colons, exclamation marks and quotation marks (Ferreiro & Pontecorvo, 1999, p. 549). They also report large differences regarding the use of punctuation in the children’s texts. One boy, for example, wrote a vivid story full of details and with an interesting story line but without any punctuation marks (Ferreiro & Pontecorvo, 1999, pp. 551-553). Another child used considerably more punctuation but sometimes at unexpected places such as in the middle of a sentence (e. g. he will arrive more, fastly). Hence, there seem to be substantial 72 2 A Comprehensive Model of Writing Competence for Young EFL Learners differences in how primary school learners use punctuation, and its use might not be as simple for young learners as generally thought. Hall and Sing (2011) studied 7to 9-year-old L1 learners’ understanding of speech marks. They found that ‘many of the children in Years 3 and 4 in this study clearly did not have a clear idea of the boundary between speech and other parts of the text’ (Hall & Sing, 2011, p. 89). The concept of direct speech and, hence, also the setting of quotation marks, were difficult for many of them (Hall & Sing, 2011). Therefore, while punctuation may not be regarded as a highly relevant and distinguishing assessment criterion at adult level, it appears to be reasonable to include and study it at primary school level (see chapter 3.2.1). 2.8.4 Relationship of the research findings to the CEFR Since this study aims at aligning the learners’ EFL writing competence to the CEFR language levels (Council of Europe, 2001), it appears to be important to discuss the research findings presented above in relation to the CEFR language level descriptors. As illustrated in the model of writing competence for young EFL learners in Figure 2.2, the CEFR scales are not used as rating scales, but rather as a framework for classifying the learners’ competence and as a basis for comparing the results of this study to the national standards for foreign language learning ( EDK , 2011), the cantonal curriculum ( BKS , 2018) and to other research studies. However, in order to facilitate the alignment of the learners’ competence to the CEFR language levels, the rating scales used for assessing the learners’ EFL writing competence were developed in close relationship to the CEFR (see chapter 3.2.1 for more details). Since the CEFR was not specifically developed for young learners (Council of Europe, 2001, p. 217), only those CEFR descriptors were considered that were marked as relevant or partially relevant for young learners aged 11-15 in Szabo (2018). A compilation of the complete set of descriptors relevant for this study can be found in . This chapter briefly summarises the CEFR descriptors for levels A1 to B1 that relate to the dimensions of text quality presented in the preceding chapters. It compares the descriptors to the research findings, identifies some issues to be considered when developing rating scales for EFL writing in a young learner context, formulates some research desiderata and summarises the comparison in Appendix A. 73 2.8 The writing product Communicative effect The CEFR does not contain any scales that specifically focus on the communi‐ cative function of a text. Even though there are different scales for written communicative language activities such as creative writing or correspondence (see Appendix A, Table A1), they do not specify the quality of the communicative effect of these texts, e. g. how interesting or entertaining a story is, or how persuasive an argument is. There is, however, a scale for propositional precision, which is defined as ‘the ability to formulate thoughts and propositions so as to make one’s meaning clear’ (Council of Europe, 2001, p. 128). With regard to this dimension, learners at level A1 are expected to be able to ‘communicate basic information about personal details and needs of a concrete type in a simple way’ (see Appendix A, Table A2, propositional precision, level A1). A learner at level A2 / A2+ is described as being able, in the context of familiar topics and still in a simple way, to communicate what he or she wants to say but may have to compromise the message if the topic is less familiar (see Appendix A, Table A2, propositional precision, level A2 / A2+). At level B1, a learner should generally be able to ‘express the main point he / she wants to make’ and communicate simple, straightforward information (see Appendix A, Table A2, propositional precision, level B1). These descriptors, however, only focus on transmitting information. Further text functions such as persuading, explaining, entertaining, instructing or inspiring seem to be missing. Since there also seems to be no research available that gives a descriptive account of whether and how young EFL learners are able to create a communi‐ cative effect in or through text composition, the rating scale for communicative effect had to be developed in reference to the learners’ texts from the piloting phase (see chapter 3.2.1). Clearly more research seems necessary to shed more light on this aspect. Coherence Even though there is a CEFR scale for coherence and cohesion, descriptors for coherence are largely missing at levels relevant for young language learners and beginners (see Appendix A, Table A2, coherence and cohesion). Only from level A2+ onwards do such descriptors exist. The descriptors ‘… to tell a story or describe something as a simple list of points’ see Appendix A, Table A2, coherence and cohesion, level A2+) or ‘can make simple, logical paragraph breaks in a longer text’ (see Appendix A, Table A2, coherence and cohesion, level B1) might represent certain aspects of coherence. Furthermore, the descriptor ‘shows awareness of the conventional structure of the text type concerned, when communicating his / her ideas’ (see Appendix A, Table A2, thematic 74 2 A Comprehensive Model of Writing Competence for Young EFL Learners development, level B1) relates to the text structure and, hence, the coherence of a text. However, descriptors for coherence at the lower language levels do not exist. Also in terms of research, there seem to be no findings available about the extent to which young EFL learners are able to write coherent texts and what the particular characteristics of their texts are in terms of coherence. Some guidance may be derived from young EFL learners’ oral narratives (Butler & Zeng, 2014), but clearly more research and a detailed analysis of the coherence of texts written by young EFL learners appear necessary. Cohesion The CEFR describes cohesion at levels A1 to B1 as moving from the linking of words and groups of words to linking simple and later longer sentences within a text (see Appendix A, Table A2, coherence and cohesion). In terms of different types of cohesive devices, the level A1 describes the use of very basic linear connectors such as the additive and temporal conjunctions and and then. Level A2 adds causal and adversative conjunctions such as because and but (see Appendix A, Table A2, coherence and cohesion, level A2) and ‘the most frequently occurring connectors’ (see Appendix A, Table A2, coherence and cohesion, level A2+). Research by Yasuda (2019) showed that besides the cohesive devices men‐ tioned as examples in the CEFR , young EFL learners also use personal pro‐ nouns, demonstratives, comparatives and lexical cohesion such as reiteration (repetition of the same word) or hyponymy (fruit / banana). Thus, in order to better represent the range of cohesive devices and reference words, it might be desirable to additionally mention lexical cohesion, personal pronouns, demonstratives and comparatives in corresponding rating scales (see chapter 3.2.1 for how this was implemented). However, since the available research about cohesion is not specifically linked to the CEFR language levels (except that Yasuda, 2019, mentions that the participants in her study were approximately at level A1), specific research by CEFR language levels might be necessary to find out more precisely what kind of cohesive devices and references words can be expected of young EFL learners at different language levels. Sociolinguistic appropriateness Sociolinguistic appropriateness in the CEFR is described as moving from establishing basic social contact (level A1), handling very short social exchanges (level A2), socialising simply but effectively (level A2+) to acting appropriately according to salient politeness conventions (level B1) (see Appendix A, Table 75 2.8 The writing product A3, sociolinguistic appropriateness). A learner on level A1 is described as using the simplest everyday polite forms such as greetings, farewells and expressions such as please, thank you and sorry, whereas a learner on level A2 should be able to ‘make and respond to invitations, suggestions, apologies etc.’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A2). A learner on level A2+ should additionally be able to make and respond to requests and to ‘express opinions and attitudes in a simple way’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A2+). These descriptors seems to be congruent with the findings by Yasuda (2019) who found that the learners in her study, whose general language level was approximately A1, used elements such as Dear …, hello, thank you and bye bye more frequently than they made requests and offers or expressed gratitude. However, since this research is also not specifically linked to the CEFR language levels, an analysis by language level might provide further information on how and to what extent young EFL learners at different language levels can establish social contact in or through writing. Vocabulary range and control In terms of vocabulary, the CEFR provides scales for range and control (see Appendix A, Table A4). At level A1, the vocabulary range is described as limited to basic words and phrases that are ‘related to particular concrete situations’ (see Appendix A, Table A4, vocabulary range, level A1). At level A2, it expands so that it becomes possible to express basic communicative needs and to conduct routine everyday transactions (see Appendix A, Table A4, vocabulary range, levels A2 and A2+). At level B1, the vocabulary is large enough to deal with most topics of everyday life with some circumlocutions (see Appendix A, Table A4, vocabulary range, level B1). This focus on vocabulary from a familiar and everyday context at levels A1 and A2 appears to accord well with the research by Llach and Gómez (2007), who found that the young EFL learners mainly used everyday vocabulary from familiar semantic fields in their texts. The CEFR descriptors for vocabulary control do not seem to be very specific at lower levels. While there is no descriptor for level A1, learners at level A2 should be able to ‘control a narrow repertoire dealing with concrete everyday needs’ (see Appendix A, Table A4, vocabulary control, levels A2 and A2+). What exactly the term control refers to at this level, however, is not further specified (see Council of Europe, 2001; Szabo, 2018). The research studies by Lasagabaster and Doiz (2003), Lindgren and Stevenson (2013) and Llach and Gómez (2007) seem to provide important insights that may help in this respect. According to their findings, control may, for example, be connected to a visible influence of the 76 2 A Comprehensive Model of Writing Competence for Young EFL Learners first language, such as borrowings or substitutes (which also seems to be related to vocabulary range, as it could be a question of the availability of language resources). Since the research studies are not specifically linked to the CEFR , further research by language level may be necessary to provide information for specifying the CEFR descriptors for vocabulary control at levels A1 and A2, and corresponding rating scales. Grammar: syntax and morphology In the CEFR , descriptors for grammar can be found under general linguistic range and grammatical accuracy (see Appendix A, Table A4). Syntactical complexity is described as developing from isolated words and basic expressions (level pre-A1), simple expressions and ‘basic structures in one-clause sentences with some omission or reduction of elements’ (level A1) and basic sentence patterns and memorised phrases (level A2) to ‘enough language to get by’ with some circumlocutions (level B1) (see Appendix A, Table A4, general linguistic range). Grammatical accuracy is described as moving from ‘only limited control of a few simple grammatical structures and sentence patterns’ (level A1) to the correct use of some simple structures while still systematically making basic mistakes (level A2 / A2+), to a reasonably accurate use of frequently used patterns (level B1) (see Appendix A, Table A4, grammatical accuracy). Generally, these descriptors appear to accord well with the research findings by Hasselgreen and Sundet (2017) and Raaen and Guldal (2012), in that they indicate an increasing complexity and correctness of grammatical structures, while recognising that this increase might not be a linear process, and that some simple structures may be used correctly while others might still be erroneous (Hasselgreen & Sundet, 2017; Lasagabaster & Doiz, 2003). As with vocabulary, a certain transfer of L1 structures to the foreign language seems to take place in order to compensate for limited L2 proficiency, an aspect that is not represented in the CEFR descriptors. Besides the analysis of the use of past tense forms at different CEFR language levels by Hasselgreen and Sundet (2017), none of the other research studies is linked to the CEFR language levels. Thus, further research linked to the CEFR language levels might be able to shed more light on the type of grammatical structures used by young EFL learners at different language levels, and on their frequency of occurrence. Orthography The CEFR descriptors for orthographic control describe a learner at level A1 as being able to copy familiar words and short phrases, and spell personal details such as the address or nationality (see Appendix A, Table A4, orthographic 77 2.8 The writing product control, level A1). At level A2 / A2+, the learner should be able to copy short sentences and ‘write with reasonable phonetic accuracy (but not necessarily fully standard spelling) short words that are in his / her oral vocabulary’ (see Appendix A, Table A4, orthographic control, level A2 / A2+). At level B1, spelling should be ‘accurate enough to be followed most of the time’ (see Appendix A, Table A4, orthographic control, level B1). The descriptor ‘can write with reasonable phonetic accuracy’ (orthographic control, level A2) appears to accord well with the research findings that young EFL learners’ orthography often reflects oral pronunciation and is influenced by L1 pronunciation and spelling (Lasagabaster & Doiz, 2003; Llach & Gómez, 2007). The research, however, is not specifically linked to the CEFR language levels. Since there also seem to be differences in terms of orthography between different age groups that were exposed to the target language to a similar extent (Lasagabaster & Doiz, 2003), further descriptive research among different age groups and by CEFR language levels might provide information that could help to decide whether the same or different descriptors or rating scales for young EFL learners and for older students should be used. Punctuation In the CEFR , punctuation is integrated in the scale for orthographic control. As can be seen in Appendix A, Table A4, learners at level A1 should be able to use basic punctuation such as full stops and questions marks. For the levels A2 and A2+, a descriptor for punctuation is missing and at level B1, punctuation is expected to be ‘accurate enough to be followed most of the time’ (see Appendix A, Table A4, orthographic control, level B1). Since no research from a young EFL learner context appears to be available, further research such as that by Ferreiro and Pontecorvo (1999) or Hall and Sing (2011) from an L1 context might be necessary to determine more precisely what can be expected of young EFL learners in terms of punctuation. 78 2 A Comprehensive Model of Writing Competence for Young EFL Learners Text quality Sources a Corresponding CEFR descriptors b Comparison, observations Research desiderata Pragmatic Communicative effect - Propositional precision No research from a young EFL learner context available. The CEFR descriptors for propositional precision mainly focus on conveying information. Further text functions may have to be considered (e. g. persuading, explaining, entertaining or in‐ structing). Research might be able to provide infor‐ mation on whether and to what extent young EFL learners are able to write texts that fulfil their communicative pur‐ pose. It may also be able to show what means the learners use to create a com‐ municative effect. Coherence - Coherence and cohesion Thematic development No research from a young EFL learner context available. At the levels A1 and A2, descriptors for coherence are largely missing in the CEFR. Research might provide insight into whether and how well young EFL learners are able to structure different types of texts. Cohesion Yasuda (2019) Hasselgreen and Sundet (2017) Coherence and cohesion The research studies are not specifically linked to the CEFR language levels, except that Yasuda (2019) mentions that the par‐ ticipants in her study were approximately at level A1. The CEFR only mentions four examples of cohesive devices explic‐ itly, namely and, then, because and but. Research shows that young EFL learners (besides additive, temporal, causal and ad‐ versative conjunctions) also use personal pronouns, demonstratives, comparatives and lexical cohesion such as reiteration (repetition of the same word) and hy‐ ponymy (fruit / banana). A specific analysis by language level might provide insight in the type of cohesive devices and reference words young EFL learners use at different lan‐ guage levels. 79 2.8 The writing product Text quality Sources a Corresponding CEFR descriptors b Comparison, observations Research desiderata Sociolinguistic Addressing the reader Yasuda (2019) Lindgren and Stevenson (2013) Sociolinguistic appropri‐ ateness The research studies are not specifically linked to the CEFR language levels. The CEFR descriptors at the relevant language levels generally appear to accord well with the research findings. A specific analysis by language level might provide information on how young EFL learners establish social con‐ tact at different language levels. Linguistic Vocabulary Llach and Gómez (2007) Lasagabaster and Doiz (2003) Lindgren and Stevenson (2013) Vocabulary range Vocabulary control The research studies are not specifically linked to the CEFR language levels. In terms of vocabulary range (everyday vo‐ cabulary, familiar situations and topics), the CEFR descriptors at the relevant lan‐ guage levels appear to accord well with the research findings. In terms of vocabu‐ lary control, there is no CEFR descriptor at level A1 and only one at level A2. It does not seem to be fully clear what the term ‘control’ refers to at this language level. Research from a young EFL learner context shows that ‘control’ may also have to be related to the amount of vocabulary borrowed or adapted from the learners’ first language. A specific analysis by language level might be able to provide information that would help to specify the CEFR descriptors for vocabulary control at the levels A1 and A2. Grammar (syntax and morphology) Lasagabaster and Doiz (2003) Hasselgreen and Sundet (2017) Raaen and Guldal (2012) Lindgren and Stevenson (2013) Griva, Tsakiridou, and Nihoritou (2009) General linguistic range Grammatical accuracy Besides Hasselgreen and Sundet’s (2017) analysis of the use of past tense forms at different CEFR language levels, none of the other research studies is linked to the CEFR language levels. Generally, the CEFR descriptors appear to accord well with the research findings, in that they indicate an increasing complexity and cor‐ rectness of grammatical structures, while recognising that this increase might not be a linear process, and that some simple structures may be used correctly while others might still be erroneous. As with Further research that is linked to the CEFR language levels might be able to shed more light on the type of gram‐ matical structures used at the different language levels, and on their frequency of occurrence. 80 2 A Comprehensive Model of Writing Competence for Young EFL Learners Text quality Sources a Corresponding CEFR descriptors b Comparison, observations Research desiderata vocabulary, a certain transfer of L1 struc‐ tures to the L2 seems to take place in order to compensate for limited L2 proficiency, an aspect that is not represented in the CEFR descriptors. Orthography Llach and Gómez (2007) Vogt and Bader (2017) Raaen and Guldal (2012) Lasagabaster and Doiz (2003) Orthographic control The research studies are not specifically linked to the CEFR language levels. The CEFR descriptor ‘can write with reason‐ able phonetic accuracy’ (orthographic control, level A2) appears to accord well with the research findings that young EFL learners’ orthography often reflects oral pronunciation and is influenced by L1 pro‐ nunciation and spelling. Further descriptive research (e. g. in terms of influence of oral pronunciation or L1 spelling and pronunciation) that is linked to the CEFR language levels and focuses on different age groups might provide information that would help to decide whether the same or different descriptors or rating scales might have to be used for young EFL learners as for older students (see Lasagabaster & Doiz, 2003, who found that the orthography of learners of different age groups differed even if they had been exposed to the English language to a similar extent). Punctuation - Orthographic control No research from a young EFL learner context available. The CEFR descriptors for punctuation at the language levels rel‐ evant for young learners are incomplete. There is one descriptor at level A1 (can use basic punctuation, e. g. full stops, question marks), but none at level A2. Clearly more research seems necessary to find out how and to what extent young EFL learners use punctuation at the dif‐ ferent language levels. Note. a Research giving a detailed descriptive account of the quality of young EFL learners’ written texts. b see Szabo (2018) Table 2.3 Comparison of the research findings with the corresponding CEFR descriptors 81 2.8 The writing product 2.9 Learning and change After examining the different components of the model of writing competence for young EFL learners (chapters 2.3 to 2.8), the model will now be considered as a whole and be brought into relation with the topic of learning and change, or in other words, with the development of writing competence. According to Becker-Mrotzek (2004), the term writing development refers to the development of the complex skills of text production that follows the acquisition of the written language (p. 41). Since foreign language learning in Switzerland generally starts in grade 3 ( EDK , 2018), the pupils have already acquired basic writing skills in the school language when they start learning the first foreign language. Therefore, there is usually no need to develop basic literacy skills such as learning the alphabet or shaping individual letters in the EFL classroom. This chapter, hence, focuses on writing development in the sense of learning to produce texts. As Hallet (2016) emphasises, this may at first mean the production of very short and simple texts, such as three-sentence narratives or two-sentence arguments (p. 70). According to Pinter (2017), a commonly practised activity with young EFL learners of this age is the so-called guided writing, where the children use a simple text model or framework which they complete and personalise with their own ideas, and thus create their own texts. According to Pinter (2017), such text models or frameworks may, for example, encompass cards, invitations, letters, stories or posters. She argues that these genres are important because they can be used to introduce the idea of writing for an audience, and learners can begin to see that we write differently depending on who we are writing for. Guided writing activities can be motivating because they allow children to write longer pieces of text by substituting their personally relevant messages into a given frame. These products can also be displayed or taken home. (Pinter, 2017, p. 87) When the learners get older and have more language resources at their disposal, they may become able to produce freer texts, such as cartoon stories, instruc‐ tions, recipes, diary entries or class newspapers (Pinter, 2017). As already out‐ lined in chapter 2.2.3, the learning - or the development of writing competence - appears to be a multidimensional endeavour as much as the writing process itself. Moreover, writing may not only be the expression or the result of the learning that preceded it, but in itself be a learning opportunity. Language may be consolidated and moved into the long-term memory; new pieces of language may be acquired, e. g. when looking up words in a dictionary; genre-knowledge may be built up; the pupils may learn to consider the reader’s perspective; 82 2 A Comprehensive Model of Writing Competence for Young EFL Learners to control their emotions and get help when needed; to use specific writing strategies or to collaborate with peers; to plan and revise their texts; or how to handle technology. Thus, there seem to be ample opportunities for learning in a writing task, and they may relate to all elements of the model of writing competence presented in chapter 2.2.3. This multidimensional view of writing development is increasingly sup‐ ported by research findings. While various attempts have been made to describe young learners’ writing development as a series of stages (e. g. Augst et al., 2007; Bereiter, 1980), it is now increasingly assumed that writing development is a complex, multidimensional process. Bereiter (1980), when presenting his five stages of writing development, acknowledged that there are research findings, experiments and methodological approaches that ‘support the view … that there is no natural order of writing development, in the sense of a fixed sequence that all writers must go through’ (p. 89). More recently, Becker-Mrotzek and Böttcher (2014) argued that writing development does not begin with separate subskills, which are then joined together, but that writing develops simultaneously in all dimensions (p. 62). Behrens (2017) discusses in a similar way that there is much evidence to suggest that children turn to different aspects of writing at the same time, and that children already possess more comprehensive literacy skills at a young age than they were previously thought to have. Hüttis-Graff and Merklinger (2010), for example, asked five-year-old children to ‘compose’ texts by dictating them to an adult, who immediately wrote the texts down and spoke while writing to make the children aware of the writing process. By dictating, the children ‘composed’ texts even though they could not yet write (Hüttis-Graff & Merklinger, 2010). The study showed that the pre-school children used genre-specific structures in their texts, revised and specified formulations and vocabulary, used direct speech with reporting clauses or created tension in their texts (Hüttis-Graff & Merklinger, 2010; Merklinger, 2010). Hüttis-Graff and Merklinger (2010) report that a boy, Andre, was well aware of the potential reader of his story and that he consciously considered genre-specific elements such as the title and the setting of the scene: First of all, one should know what the story is called: The lion cannot write. … Now one should also know what the story is about. … About a lion and he wants to be so much in love (p. 192). Andre also revised the text he dictated: It is about a lion. … ‘This story [is about a lion]’ would be better; and he corrected an incorrect word ending, thus showing awareness of the language (Hüttis-Graff & Merklinger, 2010, pp. 192-193). Therefore, if even pre-school children display such abilities, and awareness and knowledge about genres and language, it appears central to use and build on this. A key aspect to consider, however, is that young EFL learners may not 83 2.9 Learning and change always be able to display or use these abilities and resources when composing texts because of their limited language resources, and since the cognitive load of the many different aspects of writing might exceed their working memory capacity (see the research findings presented in chapter 2.6). Thus, when planning writing lessons, the teacher may have to select goals and tasks purposefully, provide an adequate amount of instructional scaffolding and help the learners focus their attention on those aspects that are relevant for the particular situation and writing task. In this way, as Hasselgreen, Kaledaite, Maldonado Martín, and Pizorn (2012) argue, writing may not only become a personally rewarding experience for the learners, but also ‘a major source of language development’ (p. 19). 2.9.1 Teaching EFL writing to young learners Two of the most frequently used approaches to teaching second language writing appear to be the process and the genre approach (see e. g. Hyland, 2019). This chapter briefly describes the two approaches, presents research findings on their effectiveness in a young L1 and EFL learner context, and discusses their suitability and limitations with regard to teaching EFL writing at primary school. In this study, the different elements of the process and genre approach were used as a starting point for investigating and exploring how EFL writing is currently taught in the primary schools in the Canton of Aargau (see chapter 4.2). In addition, they formed the basis for the development of the indices in the learner and teacher questionnaires that were used to investigate the influence of different educational factors on the learners’ EFL writing competence (see chapters 3.2.3, 3.4.4 and 4.3.2). Process approach As its name implies, the process approach to teaching writing makes use of what is known about the processes good writers go through when they write and the strategies they employ (Hyland, 2011). In particular, attention is centred on the cognitive processes involved in writing (Hyland, 2011). The process approach thus mainly focuses on the performance level of the model of writing competence for young EFL learners, and the learner’s executive and monitoring function (see Figure 2.2). The teaching procedure usually consists of pre-writing activities such as brainstorming ideas and planning the text, writing a draft version, getting feedback and revising the text before it is edited and published to an audience (Hyland, 2019, pp. 11-12). 84 2 A Comprehensive Model of Writing Competence for Young EFL Learners A key characteristic of the process approach is the aim of developing metacognitive awareness of the different processes involved in writing (Hyland, 2011). The learners reflect on useful writing strategies and how they can be applied, e. g. how to generate ideas or revise a text (Hyland, 2011). As Rijlaarsdam and Van den Bergh (2005) emphasise, this only rarely occurs automatically, but needs to be specifically initiated by the teacher (p. 4), which means that they should help the learners to gain distance from the actual writing task and provide opportunities to reflect on how writing works and how strategies can be applied. Another key aspect of the process approach is the role of feedback from peers and teachers (Hyland, 2011). The central function of feedback is to provide the learner with information about how well the text fulfils its purpose, what aspects need to be improved and how the learner can approach the necessary changes (Philipp, 2015). The feedback can be given either during or at the end of the writing process (Philipp, 2015). According to Parr and Timperley (2010), high-quality feedback on written products indicates how well the learner has met the aims and how the text can be improved; it is specific, based on evidence in the text and refers to both deep and surface features of text quality. The process approach to teaching writing has been widely researched, in particular in an L1 context. Several meta-analyses have investigated the effectiveness of the process approach and its individual elements (e. g. Graham et al., 2012; Graham & Sandmel, 2011; Koster et al., 2015). Graham et al. (2012), who investigated writing instruction in elementary grades, found that strategy instruction ( ES = 1.02), peer assistance when writing ( ES = 0.89), adult feedback ( ES = 0.80), pre-writing activities ( ES = 0.54), peerand self-feedback ( ES = 0.37) and the process approach in general ( ES = 0.40) had a significant positive effect on the learners’ writing skills. Graham and Sandmel (2011) specifically examined the effectiveness of process writing and focused on studies with learners from grades 1 to 12. They similarly found a significant average effect of process writing on the learners’ writing quality ( ES = 0.34). Koster et al. (2015), who investigated the effect of different types of writing instruction in grades 4-6, found that strategy instruction ( ES = 0.96), feedback ( ES = 0.88) and peer assistance during the writing process ( ES = 0.59) had a significant positive effect on the writing quality. Interestingly, they also found that the effect size of strategy instruction was higher in grade 6 than in grades 4 and 5. Some research has also been conducted in the context of learning English as a second language ( ESL ) at primary school (see for example de Oliveira & Silva, 2016; Leki, Cumming, & Silva, 2008 for an overview). Research on the effectiveness of different types of writing instruction in a young EFL learner context, however, seems to be rare. Lee and Wong (2014) investigated the 85 2.9 Learning and change application of a process-genre approach among primary school EFL learners in Hong Kong and found that both the learners’ writing skills and their motivation to write could be improved (p. 161). Arteaga-Lara (2017) conducted a qualitative action research study and investigated in what way a process-genre approach helped young EFL learners write narrative paragraphs. He found that the learners succeeded in writing well-structured paragraphs and developed an awareness of the audience and of the genre-specific aspects of narrative texts. The process approach to teaching writing seems to be an important means of developing young learners’ awareness of writing strategies and of the writing process, in particular since children tend not to plan or revise their texts unless they are instructed to do so and supported in doing it (Bereiter & Scardamalia, 1987). However, in the first years of foreign language learning at primary school, writing tasks may be small and the learners’ texts short. Therefore, it might not be possible to focus on all aspects at the same time, but the teacher may have to select the most important aspects according to the scope and nature of the actual writing task. The process approach, however, also has its limitations (Hyland, 2011). Since it primarily focuses on the writing process, it fails to account for the social purpose of writing, the required language and the role of different genres (Hyland, 2011, pp. 20-21), which all seem to be important aspects in a young EFL learner classroom (see chapters 2.3 and 2.5). Genre approach According to Hyland (2004), ‘genre pedagogies have emerged in L2 writing classes as a response to process pedagogies, as an outcome of communica‐ tive methods, and in consequence of our growing understanding of literacy’ (p. 7). Following the concept of communicative language teaching, genre-based teaching stresses the importance of writing for a particular purpose and in a specific context (Hyland, 2004), thus focusing on the target level of the model of writing competence for young EFL learners (see Figure 2.2). Writing is seen as a social discourse with the reader, and the language needed depends on the particular situation in which writing takes place (Hyland, 2004). According to Hallet (2016), generic learning should always be embedded in the context of meaningful communicative tasks. Questions like ‘What is the content and purpose of the text? ’, ‘Who is the reader, the audience? ’ and ‘In what form can the message be conveyed? ’ should be guiding questions when working with the genre approach (Hallet, 2016, p. 99). A common procedure in genre methodology is the so-called teaching-learning cycle, which distinguishes be‐ tween five different phases: building the context; modelling and deconstructing 86 2 A Comprehensive Model of Writing Competence for Young EFL Learners the text; joint construction of the text; independent construction of the text; and linking related texts (Feez, 2002, p. 65). Hyland (2004) describes the different phases as follows: In the first phase, the learners explore the purposes of the genre and the context in which it is used. In the second phase, they analyse different samples of the genre, e. g. in terms of text function, content, structure or language in order to develop an awareness of the key features of the genre. In the third phase, the learners work together to create a text, e. g. in class or in small groups, with support from the teacher. In the fourth phase, the learners write individual texts. In this phase, elements of the process approach could be included, such as planning, drafting or revising. In the final phase, the learners’ texts are compared to other texts from the same or a similar context, with the aim of showing the learners that genres are flexible and modifiable rather than rules that have to be precisely followed. In a young L1 learner context, Graham et al. (2012) and Koster et al. (2015) conducted two meta-analyses that, among other aspects, investigated the effect of the genre approach and its elements on the learners’ writing competence. Graham et al. (2012) found that setting product goals ( ES = 0.76), creativity / imagery instruction ( ES = 0.70) and text structure instruction ( ES = 0.59) had a statistically significant, positive effect on the learners’ writing skills. Koster et al. (2015), who focused on grades 4 to 6, found that text structure instruction ( ES = 0.76) significantly improved the writing quality. Research on the effectiveness of the genre approach in a young EFL learner context appears to be rare. Zarei and Khalili (2017) conducted a meta-analysis on genre-based writing instruction in an EFL context. They found that genre-based instruction had a small positive average effect on writing quality ( ES = 0.30), and that it was significantly more effective at primary school ( ES = 0.52) than at secondary school ( ES = 0.34) and university ( ES = 0.27) (Zarei & Khalili, 2017, p. 122). There was, however, only one study that focused on genre-based instruction in a young EFL learner context, compared to 21 studies at university level and 6 at secondary school level (Zarei & Khalili, 2017, p. 122). According to Hallet (2016), the genre approach has the advantage that it can be used with both younger and older learners. The use of sample texts as instructional scaffolding seems to be a crucial aspect when teaching young EFL learners, in particular since the cognitive and linguistic demands of writing in a foreign language are very high (McCutchen, 2006), see also chapters 2.5 and 2.6. Even though a text analysis with young learners might be much simpler than with teenage or adult learners, they still seem to be able to use such a text as a model or template (Hallet, 2016). Slow learners may copy much of the original text and change some key words to create their own piece of writing, 87 2.9 Learning and change whereas stronger learners may use the text model more flexibly. Especially for weaker pupils it seems of great importance to have generic scaffolds for their communicative acts: basic interactional, linguistic-discursive and textual structures which they can use to practise oral and written communication (Hallet, 2016, pp. 29-30). Like the process approach, the genre approach also seems to have its limitations. Hyland (2011), for example, argues that there may be a danger of only focusing on genres that reflect the prevailing culture or of restricting the learners’ creativity through conformity. It seems important, therefore, for those who apply the genre approach to be aware of this danger and to try to avoid it by using a variety of genres and by giving room to the learners’ creativity. An aspect that neither the genre nor the process approach seems to specifi‐ cally take into account is the young EFL learners’ limited language skills and resources (see chapter 2.5.1). Since this aspect appears to be of paramount importance when teaching EFL writing to young learners, there seems to be a clear need for the development of a teaching approach that specifically considers this aspect. An approach that seems to have great potential in this respect might be the so-called pre-while-post framework, which guides the learners from an input to an output, for example from reading to writing, and regards text comprehension as a key scaffold for developing writing skills (Bader & Trüb, 2020). This approach, however, does not yet seem to have been researched, and thus specific research may be necessary in order to assess its effectiveness for teaching EFL writing to young learners. 2.9.2 Young EFL learners’ extracurricular use of English Another aspect that is not directly related to classroom instruction, but appears to increasingly gain relevance, is the learners’ extracurricular use of the English language. It is concerned with ‘any type of situation in which learners come in contact with or are involved in English outside the walls of the English classroom’ (Sundqvist, 2009, p. 66). Several studies found that the learners’ extra‐ curricular encounters with the English language had an effect on their learning at school. Sundqvist (2009), for example, in a study among ninth-grade EFL learners, found that it was positively correlated with the learners’ speaking skills and vocabulary. Sylvén and Sundqvist (2012) found similar results with regard to digital games and fifth-grade EFL learners’ vocabulary and their listening and reading skills. In terms of EFL writing, Olsson (2011) found a statistically significant correlation of 0.6 between 16-year-old EFL learners’ extracurricular use of English and their writing proficiency. In particular the frequency of 88 2 A Comprehensive Model of Writing Competence for Young EFL Learners reading, writing and watching English TV programmes or films was related to the learners’ writing proficiency (Olsson, 2011). This study examines whether the extracurricular use of the English language also influences primary school EFL learners’ writing competence (see chapter 4.3.1). 2.10 Research questions After the presentation and discussion of the model of writing competence for young EFL learners and its different components, the chapter now turns to specifying the research questions of this study. As mentioned in the introduc‐ tion, the study consists of three parts, which try to shed light on EFL writing at primary school in terms of the learners’ writing competence, the teachers’ and learners’ perception of EFL writing and current teaching practices, and different individual and educational factors being predictors of the learners’ writing competence. 2.10.1 Part I: The young EFL learners’ writing competence Part I of the study is concerned with the question of how proficient the pupils are in EFL writing in grade 6 at the end of primary school. This part is subdivided into three sections that try to describe the learners’ EFL writing competence from three different perspectives: First, the study investigates whether and how well the learners reach the requirements for EFL writing as stated in the Swiss national standards for foreign language learning ( EDK , 2011) and in the curriculum of the Canton of Aargau ( BKS , 2018). Both documents define level A1.2 as the minimum requirement for EFL writing at the end of primary school ( BKS , 2018; EDK , 2011). Hence, the study tries to answer the following question: (RQ I.1) What CEFR language level do the learners reach in EFL writing at the end of primary school, and what percentage of learners achieves the minimum requirements as stated in the Swiss national standards and the cantonal curriculum? Secondly, the study examines whether differences in terms of EFL writing competence can be observed between the groups of learners who are about to enter the different educational tracks at lower secondary school ( ISCED 2 according to UNESCO , 2012). This is done in order to provide lower secondary school teachers with information about what writing competence the learners 89 2.10 Research questions may have when they enter secondary school. The second section, therefore, addresses the following question: (RQ I.2) What differences in terms of EFL writing competence can be observed when the learners are grouped according to their future educational track at lower secondary school? Thirdly, the study aims at giving a detailed account of the different character‐ istics and qualities of the learners’ texts. A descriptive analysis of sample texts is intended to provide a more comprehensive picture of the learners’ writing competence than the first, quantitative analysis is able to give. Since descriptive findings about the communicative effect and coherence of texts written by young EFL learners are rare or do not exist (see chapter 2.8), these two dimensions are examined in more detail. This leads to the following research questions: (RQ I.3) What are the characteristics and qualities of young EFL learners’ texts at different language levels? (RQ I.4) How do young EFL learners create a communicative effect in narrative texts? (RQ I.5) What are the characteristics of coherence in young EFL learners’ narra‐ tive texts? 2.10.2 Part II: Current teaching practices and the learners’ perception of EFL writing Part II is an explorative study that investigates how primary EFL writing is taught in the Canton of Aargau and what the learners’ perception of text composition in the foreign language is. Due to the limited scope of the study, this part only focuses on information provided by the teachers and learners rather than direct classroom investigation. It consists of two sections: the teachers’ and the learners’ perspective. The first section is concerned with an inquiry among primary school EFL teachers about how they teach EFL writing: the role and status they assign to writing, the goals they pursue and the methodology they use. The study tries to answer the following questions: (RQ II.1) What role do primary EFL teachers assign to writing compared to other skills and teaching components such as listening, speaking, reading, use 90 2 A Comprehensive Model of Writing Competence for Young EFL Learners of strategies, language and cultural awareness, spelling, grammar and vocabulary? (RQ II.2) What aims do primary EFL teachers pursue when teaching writing? (RQ II.3) How frequently is text composition practised in class? (RQ II.4) What aspects of text quality do the teachers discuss or address in class? (RQ II.5) What types of writing tasks and genres are used in class? (RQ II.6) What types of pre-writing activities, scaffolding and feedback do the teachers employ when teaching EFL writing? (RQ II.7) What types of writing strategies are used in the classroom? (RQ II.8) How do the teachers assess written products? The second section deals with the learners’ perception of EFL writing at primary school. It explores the learners’ personal view of EFL writing, their preferences and the types of procedures they experience in class when they compose texts. Since young EFL learners often face considerable challenges when writing in a foreign language (see e. g. Griva et al., 2009), the study also investigates what types of difficulties the learners encounter and what strategies they use to overcome them. The following questions will be addressed: (RQ II.9) How much importance do the learners assign to learning to write in the English language? (RQ II.10) How much confidence do the learners have in their ability to write in English? (RQ II.11) What do the learners like and dislike about writing in English? (RQ II.12) What topics would the learners like to write about? (RQ II.13) What procedures do the learners experience when they compose texts in class? (RQ II.14) What difficulties do the learners encounter when writing in English and what strategies do they use to overcome them? 2.10.3 Part III: Predictors of the learners’ EFL writing competence The third and last part of the study examines to what extent different factors are predictors of the learners’ EFL writing competence. It is divided into two sections that focus on individual and educational factors. The first section 91 2.10 Research questions investigates the influence of the learners’ self-efficacy and extra-curricular use of English on their writing competence: (RQ III.1) Is the self-efficacy of young EFL learners a predictor of their EFL writing competence? (RQ III.2) Is the young EFL learners’ extra-curricular use of English a predictor of their EFL writing competence? The second section focuses on the impact of different teaching factors on the learners’ EFL writing competence: (RQ III.3) Are teaching factors such as the frequency of text composition in class, the importance assigned to reading, the frequency of vocabulary learning, and the role assigned to pre-writing activities, instructional scaffolding, feedback and orthography predictors of the learners’ EFL writing competence? 92 2 A Comprehensive Model of Writing Competence for Young EFL Learners 3 Methods 3.1 Research design 3.1.1 Overview This study comprises three parts, which focus on three different aspects of EFL writing at primary school, namely the learners’ EFL writing competence, current teaching practices and predictors of the learners’ writing competence (see Figure 3.1). Figure 3.1 Overview of the three parts of the study In order to answer the research questions presented in chapter 2.10, different types of data and research methods were required. The basis for parts I and III of the study was the measuring of the young EFL learners’ writing competence. Since writing competence is a latent construct that cannot be observed directly, conclusions about writing competence are usually drawn from the learners’ performance in a writing task (see e. g. Grotjahn, 2017; Weigle, 2002). The research may focus on the writing process, on the final product, or on both aspects. Either approach appears to provide useful insights. In order to be able to answer the research questions presented in chapter 2.10, it seemed suitable in this study to focus on the writing product for measuring the learners’ writing competence. The study applied a multi-level approach to gauging the learners’ writing competence that used tasks which were able to measure writing competence at several CEFR language levels (Porsch, 2010; Porsch & Köller, 2010), see chapter 3.2.2. Text quality was determined with various semi-holistic rating scales that provided an overall assessment of different dimensions of text quality such as communicative effect / creativity, coherence and cohesion, vocabulary, complexity and correctness of syntax and grammar or orthography (see chapter 3.2.1). The majority of texts was rated independently by two raters, and a many-facet Rasch analysis was conducted to adjust the scores for difficulty of the task, rater severity and difficulty of the rating criteria (see chapter 3.4.3). In addition, a standard setting was conducted to align the learners’ writing scores to the CEFR language levels (see chapter 3.5). For part II of the study, a survey encompassing a learner and a teacher questionnaire was conducted in order to provide insights into current teaching practices and into the learners’ perception of EFL writing at primary school, (see chapter 3.2.3). This data was complemented with information from learner interviews (see chapter 3.2.4) in order to allow for data triangulation and specification. The same questionnaires were used to collect quantitative data with regard to different individual and educational factors that were assumed to be predictors of the learners’ writing competence (see chapter 3.2.3). This data was used for different statistical analyses in part III of this study (see chapter 4.3). 3.1.2 Evidence-centred assessment design The study followed the concept of an evidence-centred assessment design ( ECD ), as presented by Mislevy, Almond, and Lukas (2003). ECD is ‘an approach to constructing educational assessments in terms of evidentiary arguments’ (Mislevy et al., 2003, p. i) and aims at ensuring that ‘the way in which evidence is gathered and interpreted is consistent with the underlying knowledge and purposes the assessment is intended to address’ (Mislevy et al., 2003, p. 2). Since the learners’ EFL writing competence is a latent construct that cannot be observed directly, a concept for test validation such as the ECD appears necessary in order to allow for inferences to be drawn from the test scores about the learners’ writing competence. A key element of the ECD is the so-called conceptual assessment framework ( CAF ), which consists of six models that illustrate the different elements of the assessment and their relationships to each other, namely the student, evidence, task, assembly, presentation and delivery model (see Figure 3.2). It shows how 94 3 Methods the latent construct ( EFL writing competence) is operationalised in evidence rules (rating criteria) and in tasks that bring out the learners’ performance. Figure 3.2 Conceptual assessment framework by Mislevy et al. (2003, p. 5) The student model ‘defines one or more variables related to the knowledge, skills, and abilities we wish to measure’ (Mislevy et al., 2003, p. 6). For this study, the different variables or constructs of the student model have been defined in chapter 2. The evidence model gives ‘detailed instructions on how we should update our information about the student model variables given a performance in the form of examinees’ work products from tasks’ (Mislevy et al., 2003, p. 8, emphasis in the original), and consists of evidence rules and a measurement model. The evidence rules ‘describe how observable variables summarize an examinee’s performance in a particular task from the work product that the examinee produced for that task’ (Mislevy et al., 2003, pp. 8-9, emphasis in the original). The evidence rules in this study, namely the rating scales used for assessing the quality of the learners’ texts, and their relationship to the student model (which is part of the assembly model, see below), are presented in chapter 3.2.1. The measurement model, on the other hand, ‘provides information about the connection between student model variables and observable variables’ (Mislevy et al., 2003, p. 9, emphasis in the original). Chapter 3.4.3 describes the summary scoring process and the many-facet Rasch analysis that were used to obtain the learners’ final writing scores. The chapter also presents different reliability and validity measures that were considered as evidence to legitimise 95 3.1 Research design the drawing of conclusions from the final writing scores on the learners’ EFL writing competence. An aspect that is not explicitly mentioned in the CAF is the role the raters play in the rating process. Since the selection and training of the raters and the reliability of their ratings appear to be a central aspect of the assessment design, they are specifically discussed in chapters 3.2.1 and 3.4.3. The task model of the CAF describes ‘how to structure the kinds of situations we need to obtain the kinds of evidence needed for the evidence models. They describe the presentation material that is presented to the examinee and the work products generated in response (Mislevy et al., 2003, p. 10, emphasis in the original). The two writing tasks used in this study to gauge the learners’ EFL writing competence, and their relationship to the student and evidence model (which is part of the assembly model, see below), are presented in chapter 3.2.2. The assembly model describes ‘how the student models, evidence models, and task models must work together to form the psychometric backbone of the assessment’ (Mislevy et al., 2003, p. 11). This relationship between the different models appears crucial for determining the validity of the whole assessment design. Since the development of the rating scales and the writing tasks used in this study are closely linked to each other and to the student model, the assembly model is not dealt with separately, but directly included in the discussion of the evidence and task model (see chapters 3.2.1 and 3.2.2). The presentation model describes ‘how the tasks appear in various settings, providing a style sheet for organizing the material to be presented and captured’ (Mislevy et al., 2003, p. 12). It concerns the way the assessment is delivered (e. g. online or as a test booklet), or how information is presented (Mislevy et al., 2003). Thus, chapter 3.4.2 presents how the data collection was implemented, and a test administration script was used to ensure that the collection of data was carried out in the same way in all classes. Lastly, the delivery model describes ‘the collection of student, evidence, task, assembly, and presentation models necessary for the assessment and how they will work together. It also describes issues that cut across all of the other models’ (Mislevy et al., 2003, p. 13). The whole of chapter 3 is thus considered as the delivery model. Besides the previously mentioned aspects, it also includes information about the sampling method and participants (chapter 3.3), ethical considerations (chapter 3.4.1), and information about the qualitative data col‐ lected in questionnaires (chapters 3.2.3 and 3.4.4) and interviews (chapters 3.2.4, 3.4.5 and 3.6.2), and about the descriptive text analyses (chapter 3.6.1). 96 3 Methods 3.2 Data collection instruments This chapter presents the data collection instruments that were used in the study and describes how they relate to the empirical findings and theory presented in chapter 2. It reports on their development and contents and describes the different measures taken to ensure their quality. 3.2.1 Rating scales and rater training General rating method A first important decision when developing rating scales is to determine the type of rating scale that is to be used for assessing the learners’ writing competence (Weigle, 2002). A quick rating on a global scale that represents the reader’s general impression of a text is usually referred to as a holistic rating (see e. g. Grotjahn & Kleppin, 2017c). A semi-holistic rating requires the reader ‘to consider the student text as a whole, but only regarding a specific dimension of writing’ (Schipolowski & Böhme, 2016, p. 8), such as content, organisation or vocabulary range. Semi-holistic rating scales are sometimes also referred to as analytic scales (e. g. Grotjahn & Kleppin, 2017c; Weigle, 2002). However, in order to clearly distinguish them from an analytic rating that uses specific, often dichotomous rating criteria (e. g. yes / no or present / missing) (Schipolowski & Böhme, 2016) or discourse analytic measures (e. g. average word length, percentage of error-free t-units or number of orthographic errors) (Knoch, 2009), the term semi-holistic rating is preferred. A semi-holistic rating approach has been applied in this study, for several reasons. First, a holistic rating would not have provided enough information for answering the research questions RQ I.3-I.5 (see chapter 2.10), which are concerned with a detailed analysis of different dimensions of text quality. Secondly, it seemed apparent that the use of an analytic rating approach that uses highly specific and often dichotomous criteria would not be able to appropriately operationalise the multifaceted construct of EFL writing as presented in chapter 2 (see also McKay, 2006, p. 267). Furthermore, it appeared central that the rating method used in the study should conform with competency-based language teaching, which focuses on what the learners can achieve with the language rather than their deficiencies (see chapter 1.2). A semi-holistic rating approach with carefully worded rating scales for different dimensions of text quality appeared to fulfil this criterion better than an analytic approach that would have focused, among other aspects, on counts of errors or perecentages of correctly formulated sentences. Since many cantons 97 3.2 Data collection instruments in Switzerland are currently introducing the concept of competency-oriented assessment (see e. g. BKS , 2020; VSA , 2016), an analytic rating approach might have led the teachers into a very different direction than that intended by competency-oriented assessment. Thus, it appeared important to consider the possible washback effect the rating method would have in the schools. Development of the rating scales According to Weigle (2002), the rating criteria are a central aspect of the validity of an assessment, since they reflect the developer’s implicit or explicit definition of the ability to be measured. Hasselgreen et al. (2012) argue that ‘assessment criteria for writing must reflect the consensus of what good writing is. And descriptors based on these criteria must reflect the age and ability of the writers for whom they are being developed’ (p. 20). Considerable attention was thus paid to the development of the rating scales used to assess the learners’ EFL writing competence. The development of these scales is based on the construct definition of writing presented in chapter 2.1 and relates to the empirical findings about the qualities of young EFL learners’ texts presented in chapter 2.8. Furthermore, genre-specific aspects were taken into account (see chapter 2.3.2) as well as the CEFR descriptors considered as relevant or partially relevant for young language learners aged 11 to 15 (Szabo, 2018). Existing rating scales that had been developed to assess young EFL learners’ writing competence and that were linked to the CEFR , were reviewed (Brock, 2015; Hasselgreen et al., 2012). They could not be directly adopted since they did not fully match with the intended genres and tasks, but they served as helpful guidance when formulating and specifying the scales for this study. In chapter 2.1, writing was defined as the complex activity a writer, or writers, engage in in order to create a written product which fulfils a specific function, and writing competence was defined as the writer’s ability to effectively and responsibly engage in real-world writing tasks by using the available personal and external resources. Such a construct definition of writing seems to require an assessment design that allows the writer to display this competence, and measurement tools that are able to capture it. Therefore, with regard to developing the rating scales, it seemed central for the quality of the written product to be assessed with a balanced consideration of pragmatic, sociolinguistic and linguistic aspects (see also chapter 2.8). Figure 3.3 shows the content of the different rating scales developed for this study. The pragmatic dimension is represented in the scales for coverage, communicative effect/ creativity, level of detail, coherence and cohesion. The sociolinguistic dimension is represented in the scale for genre-specific elements of an e-mail, and the linguistic dimension in the scales for complexity and correctness of syntax and grammar, vocabulary range, orthography and punctuation. 98 3 Methods Figure 3.3 Content of the rating scales for the genres e-mail and story Since the study aimed at aligning the learners’ writing competence to the CEFR language levels, the descriptors for the rating scales were, as far as possible, developed in close relation to the CEFR language level descriptors considered as relevant or partially relevant for young language learners aged 11 to 15 (Szabo, 2018). Based on the Swiss national standards for foreign language learning, which define level A1.2 as the minimum requirement for EFL writing to be reached by the end of primary school ( EDK , 2011), a 5-point rating scale was created with level A1.2 as its midpoint: 0 1 2 3 4 Below Approx. level A1.1 Approx. level A1.2 Approx. level A2.1 Above Figure 3.4 5-point rating scale for assessing EFL writing at levels A1 and A2 While it was possible for most of the linguistic rating scales to be developed in reference to the CEFR descriptors, many of the pragmatic and sociolinguistic rating scales could not be based on existing scales, because corresponding CEFR descriptors did not exist, or only incompletely, and since also genre-specific aspects had to be considered. The first genre, correspondence, was implemented in the form of an informative e-mail. Thus, the scale for coverage assessed to what extent the recipients were provided with the necessary information, and the scale for level of detail focused on how detailed this information was. 99 3.2 Data collection instruments Dissertation An Empirical Study of EFL Writing at Primary School Ruth Trüb 72 06.12.2021 0 Below 1 Approx. level A1.1 Coverage Name, age, school subjects and reasons, two hobbies, questions Does not (or in a very limited way) respond to the writing prompt or not enough assessable language. Responds to some aspects of the writing prompt. Three or more aspects are missing. Level of detail Not able to convey very basic information or not enough assessable language. Conveys very basic information without much detail. Genre-specific elements of an e-mail Greeting; addressing the reader directly, e.g. ‘Sophie, do you...? ’; asking for information; concluding sentence, e.g. asking for a reply; complimentary close; signature; P.S.; smiley No genre-specific elements or not enough assessable language. The text addresses the reader in a very limited way. There are only 1-2 typical elements of an e-mail. Opening and/ or closing are missing. Coherence (logical organisation) Text almost completely incoherent or not enough assessable language. The topics sometimes change unexpectedly from one sentence to the next. The same topic may be dealt with in different parts of the text (clear disruption). Cohesion (linguistic connectivity) Repetition of key words, connectors (and, but, or, because, then,…), pronouns (he, she, they, it,...), demonstratives (this, that, there,...), comparatives (same, another, more,...) No linking of words and phrases, only isolated chunks of language or not enough assessable language. Only few and very basic cohesive devices or reference words are used to link words and sentences (e.g. and). The text mainly consists of isolated phrases and sentences. Complexity of syntax and grammar Not enough assessable language. Uses very simple and partly incomplete sentences and very simple grammatical structures, mainly in formulaic expressions. Correctness of syntax and grammar Not enough assessable language. Only very limited control and many inconsistencies. Some reduction or omission of elements. Vocabulary range Most of the text written in another language or not enough assessable language. Basic chunks and limited vocabulary, some words and/ or phrases in another language. Orthography Text hardly comprehensible or not enough assessable language. Only very limited control and many inconsistencies. Comprehension sometimes impaired. Punctuation No or almost no use of punctuation. Limited use of basic punctuation (e.g. full stops, question marks), sometimes incorrect or missing. Table 3.1 Rating scales for the e-mail (adapted from Brock (2015); Hasselgreen et al. (2012); Szabo (2016)) Table 3.1 Rating scales for the e-mail (adapted from Brock (2015); Hasselgreen et al. (2012); Szabo (2016)) 100 3 Methods Dissertation An Empirical Study of EFL Writing at Primary School Ruth Trüb 73 06.12.2021 2 Approx. level A1.2 3 Approx. level A2.1 4 Above Responds to most aspects of the writing prompt. Only one or two aspects are missing. Responds to all aspects of the writing prompt. Above Responds to all aspects of the writing prompt and adds further elements. Conveys basic information with details about 1-2 aspects. Conveys basic information with details about 3-4 aspects (or 1-2 aspects in more depth). Above Conveys detailed information. The text addresses the reader in a very simple way. There are 3-4 typical elements of an e-mail. The text addresses the reader in a simple but effective way. There are 5-6 typical elements of an e-mail. Above The text has the typical form of an email. It addresses the reader in a simple but effective and appropriate way. There is a simple sequence of topics in the text. There are only few unexpected changes or some repetition. There is a simple and clear sequence of topics in the text. Above The text shows slight elaboration of the text structure, e.g. logical sequencing of different topics or use of subtopics. A small number of very basic cohesive devices and reference words (e.g. and, this, it) are used to link words and sentences. The text contains some isolated phrases and sentences. Some simple cohesive devices and reference words (e.g. and, but, because, then, or, he, she, they) are used to link words and sentences. No or only few isolated phrases and sentences. Mostly linguistically welllinked text. Above A larger variety and amount of cohesive devices and reference words are used to link words, sentences and text passages. Linguistically well-linked text. Uses simple sentences (e.g. oneclause sentences) and simple grammatical structures (e.g. plural ‘-s’), often in formulaic expressions. Uses a mixture of simple and more complex sentences and grammatical structures (e.g. simple subordinate clauses, 3rd person ‘-s’, past or future forms). Above Uses more complex and varied sentences and grammatical structures. Only limited control and many inconsistencies. Uses few simple structures correctly but still systematically makes basic errors. Above Uses some simple structures correctly but still makes basic errors. Basic vocabulary (e.g. verbs like ‘I have’, ‘I like’ or ‘I go’), enough vocabulary to write the text mainly in English (no or only few words in another language). Mainly simple but also some specific and/ or varied vocabulary that allows a slight elaboration of the text. Above Wider range of vocabulary that allows a clear elaboration of the text. Only limited control and many inconsistencies, but the text is comprehensible. Short and very common words are written with reasonable phonetic accuracy. Still makes basic errors. Above Common words are written with reasonable phonetic accuracy. Still makes errors. Mostly correct or correct use of basic punctuation (e.g. full stops, question marks), only occasionally missing OR Use of some further elements but basic punctuation sometimes incorrect or missing. Correct use of basic punctuation. Use of some further elements, e.g. exclamation mark, colon or commas for a series of words or phrases, not necessarily correct. OR Use of some difficult elements but basic punctuation sometimes incorrect or missing. Above Correct use of common punctuation. Use of some difficult elements, e.g. commas for dependent clauses or quotation marks, not necessarily correct. Kommentiert [TR1]: Tabelle geht horizontal über zwei Seiten und sollte daher auf einer Doppelseite angeordnet werden. (Die Höhe der einzelnen Zeilen in der Tabelle sollte auf beiden Seiten gleich sein, ebenso die Platzierung der Tabelle auf der Seite.) 101 3.2 Data collection instruments Dissertation An Empirical Study of EFL Writing at Primary School Ruth Trüb 74 06.12.2021 0 Below 1 Approx. level A1.1 Communicative effect / creativity E.g. has a witty ending, creates tension, captures the reader’s attention, takes a surprising twist Not enough assessable language or the text is not comprehensible. The text is comprehensible but has only a very limited communicative effect. Level of detail E.g. detailed descriptions, reasons/ explanations, emotions Not enough assessable language. Describes the scene, people and actions without much detail. Coherence (logical organisation) Setting the scene, development/ complication, resolution, story ending Text almost completely incoherent or not enough assessable language. The writer does not fully succeed in developing a storyline. The text is incomplete and sometimes unclear. Cohesion (linguistic connectivity) Repetition of key words, connectors (and, but, or, because, then,...), pronouns (he, she, they, it,...), demonstratives (this, that, there,...), comparatives (same, another, more...) No linking of words and phrases, only isolated chunks of language or not enough assessable language. Only few and very basic cohesive devices or reference words are used to link words and sentences (e.g. and). The text mainly consists of isolated phrases and sentences. Complexity of syntax and grammar Not enough assessable language. Uses very simple and partly incomplete sentences and very simple grammatical structures, mainly in formulaic expressions. Correctness of syntax and grammar Not enough assessable language. Only very limited control and many inconsistencies. Some reduction or omission of elements. Vocabulary range Most of the text written in another language or not enough assessable language. Basic chunks and limited vocabulary, some words and/ or phrases in another language. Orthography Text hardly comprehensible or not enough assessable language. Only very limited control and many inconsistencies. Comprehension sometimes impaired. Punctuation No or almost no use of punctuation. Limited use of basic punctuation (e.g. full stops, question marks), sometimes incorrect or missing. Table 3.2 Rating scales for the story (adapted from Brock (2015); Hasselgreen et al. (2012); Szabo (2016)) Table 3.2 Rating scales for the story (adapted from Brock (2015); Hasselgreen et al. (2012); Szabo (2016)) 102 3 Methods Dissertation An Empirical Study of EFL Writing at Primary School Ruth Trüb 75 06.12.2021 2 Approx. level A1.2 3 Approx. level A2.1 4 Above The text has a small communicative effect. There are a few elements that catch the reader’s attention. The text has a simple communicative effect. It contains some elements that make the story interesting or has a witty ending. Above The story has a clear communicative effect. It creates tension and has a witty ending. Describes the scene, people and actions with a few details. Describes the scene, people and actions with some details. Above Describes the scene, people and actions in detail. It is evident that the writer is trying to tell a story. There is a very simple storyline with some gaps or incoherences. The writer succeeds in telling a story with a simple and mostly coherent storyline. There are only few gaps or incoherences. Above The storyline is coherent and contains all elements of a simple narrative structure. A small number of very basic cohesive devices and reference words (e.g. and, this, it) are used to link words and sentences. The text contains some isolated phrases and sentences. Some simple cohesive devices and reference words (e.g. and, but, because, then, or, he, she, they) are used to link words and sentences. No or only few isolated phrases and sentences. Mostly linguistically welllinked text. Above A larger variety and amount of cohesive devices and reference words are used to link words, sentences and text passages. Linguistically well-linked text. Uses simple sentences (e.g. oneclause sentences) and simple grammatical structures (e.g. plural ‘-s’), often in formulaic expressions. Uses a mixture of simple and more complex sentences and grammatical structures (e.g. simple subordinate clauses, 3rd person ‘-s’, past or future forms, reporting clause for direct speech). Above Uses more complex and varied sentences and grammatical structures. Only limited control and many inconsistencies. Uses a few simple structures correctly but still systematically makes basic errors. Above Uses some simple structures correctly but still makes basic errors. Basic vocabulary (e.g. simple home and family vocabulary, simple verbs, few adjectives), enough vocabulary to write the text mainly in English (no or only few words in another language). Mainly simple but also some specific and/ or varied vocabulary that allows a slight elaboration of the text. Above Wider range of vocabulary that allows a clear elaboration of the text. Only limited control and many inconsistencies, but the text is comprehensible. Short and very common words are written with reasonable phonetic accuracy. Still makes basic errors. Above Common words are written with reasonable phonetic accuracy. Still makes errors. Mostly correct or correct use of basic punctuation (e.g. full stops, question marks), only occasionally missing OR Use of some further elements but basic punctuation sometimes incorrect or missing. Correct use of basic punctuation. Use of some further elements, e.g. exclamation mark, colon or commas for a series of words or phrases, not necessarily correct. OR Use of some difficult elements but basic punctuation sometimes incorrect or missing. Above Correct use of common punctuation. Use of some difficult elements, e.g. commas for dependent clauses or quotation marks, not necessarily correct. 103 3.2 Data collection instruments The rating scale for genre-specific aspects of an e-mail assessed the sociolin‐ guistic quality of the texts and captured elements such as whether the e-mail opened with a greeting, whether the recipients were directly addressed (e. g. Sophie and Jacob, do you …), whether questions were asked, or whether the text contained a concluding sentence (e. g. ‘Please write back soon’), a complimentary close (e. g. ‘Best regards’) or the writer’s name at the end of the e-mail. The second genre, narration, was implemented in form of a story. The rating scale communicative effect / creativity assessed to what extent the text fulfilled its entertaining function, e. g. by creating tension, by a surprising twist or a witty ending. The scale for level of detail focused on how elaborate the story was and coherence assessed the genre-specific structure of the text (setting the scene, development, complication, resolution and story ending). The rating scales for cohesion, complexity and correctness of syntax and grammar, vocabulary, orthography and punctuation were formulated on the basis of the CEFR descriptors and, where necessary, specified and adapted. In the scale for cohesion, for example, additional cohesive devices and reference words such as lexical cohesion, personal pronouns, demonstratives and comparatives were added (see chapter 2.8.4, cohesion, for a discussion of the corresponding research findings), and the scale for vocabulary was complemented with the aspect of vocabulary written in another language, which seems to be a fairly common feature in young EFL learners’ texts (see chapters 2.8.3 and 2.8.4). The rating scale descriptors were formulated in a way that tried to ensure a focus on the learners’ achievements rather than their deficiencies, thus reflecting a competency-based view of learning. Even at the lowest levels, where negatively worded descriptors could hardly be avoided, neutral and non-offending language was used as far as possible (see Table 3.1 and Table 3.2). Since the CEFR does not distinguish between levels A1.1 and A1.2, another key aspect was to find descriptors that clearly distinguished between these two levels. In order to ensure the quality of this distinction, a draft of the descriptors was discussed with various experts, tested with sample texts from the pilot study and revised based on the corresponding feedback and findings (for more details see the next section on rater training and scale revision). Combined rater training and scale revision To further ensure the validity of the rating scales, a first draft was discussed with different experts from the fields of L1 and L2 writing research, EFL teaching and English linguistics. Their feedback was used to adapt the scales before they were piloted. A first rating was conducted with 24 texts from the first pilot study. This trial run resulted in the replacement of the two rating scales syntax 104 3 Methods and grammar by the scales complexity of syntax and grammar and correctness of syntax and grammar, since the original scales did not sufficiently differentiate between complexity and correctness. This separation helped to account for the variability between the complexity and correctness of grammatical structures that was also observed by Lasagabaster and Doiz (2003) and Hasselgreen and Sundet (2017). In particular, the descriptor ‘Uses few simple structures correctly but still systematically makes basic errors’, which had been adapted from the CEFR descriptor for grammatical accuracy at level A2 (see Table A4), allowed distinctions to be made between simple structures that were used correctly and more complex structures that were used incorrectly. In addition to this adaptation, several further descriptors were reformulated, as they were found to be ambiguous. At this stage, a first draft of a support document for cases of uncertainty was developed which was intended to clarify questions that could not be captured by the rating scales (e. g. whether missing reporting clauses for direct speech should be attributed to coherence or cohesion, document available from the author on request). In addition to this preparatory work, it is important to ensure that the rating scales are used appropriately and consistently by the raters (Weigle, 2002). Thus, after the preliminary rating, two raters assessed all texts from the first pilot study (43 e-mails and 44 stories) as part of a rater training that followed the combined rater training and scale revision approach suggested by Harsch and Martin (2012). Both raters were experienced primary school EFL teachers with a high command of the English language. They noted all difficulties and uncertainties they encountered during the rating process, and inter-rater reliability was regularly monitored. These two types of information were used in combination as a basis for an in-depth discussion of the texts and the rating scales. This process helped the raters to develop a common understanding of the scales, and the insights gained from this pro fed back into the revision process. Cases of uncertainty that could not be resolved by adjusting the rating scales were added to the support document for cases of uncertainty (document available from the author on request). For some rating scales, benchmark texts were provided to support the rating process. Since one of the raters unexpectedly had to resign from the rating task after the first piloting, a new rater was introduced to the task, who was also a very experienced primary school teacher with a good command of the English language. A second pilot study was carried out in order to test the adapted writing tasks, and the texts from this pilot study were used for a second rater training before the main data collection (see chapter 3.4.3 for the final inter-rater reliability measures). 105 3.2 Data collection instruments 3.2.2 Writing tasks Together with the rating scales, the writing tasks play a decisive role in measuring the learners’ writing competence, since they operationalise the test construct (Grotjahn & Kleppin, 2017a), and are thus a key aspect of the validity of the assessment (Grotjahn & Kleppin, 2017b). This chapter shows how the construct definition of writing presented in chapter 2.1 was operationalised in the writing tasks, and in what way the specific prerequisites of young EFL learners (see chapters 2.3 to 2.9) were taken into account. It also describes how the writing tasks relate to the rating criteria presented in the preceding chapter, and what instruments were used to assess and ensure the quality of the tasks. General approach to task design If writing is regarded as a multifaceted, complex activity by which a writer produces a final product that is intended to fulfil a particular function (see chapter 2.1), it seems important in terms of construct validity (Grotjahn & Kleppin, 2017b) that this also be reflected in the tasks used for gauging the learners’ EFL writing competence. Instead of using discrete-point tasks such as completing sentences, labelling pictures or correcting jumbled sentences, which often focus on one specific aspect of language knowledge such as grammar, vocabulary or spelling (Weigle, 2002), the tasks used in this study tried to generate written products at text level and, hence, reflect a more comprehensive view of writing competence. The tasks were not aimed at measuring the learners’ resources, but at measuring their ability to use these resources for communicative purposes. Even though young EFL learners’ texts might be short, the tasks were intended to give the learners the opportunity to display their full range of abilities, including pragmatic and sociolinguistic aspects such as the ability to communicate effectively or to socialise with the reader (see McKay, 2006, p. 249). Therefore, the study followed the concept of performance assessment, which ‘involves either the observation of behavior in the real world or a simulation of real-life activity’ (Weigle, 2002, p. 46). Language level targeted With regard to the language level targeted, Porsch (2010) and Porsch and Köller (2010) distinguish between two different approaches to gauging the learners’ writing competence. The so-called uni-level approach uses writing tasks that are developed for one specific CEFR language level (Porsch, 2010). This approach has the advantage that the writing tasks can be specifically customised to a particular language level (Porsch, 2010). However, several tasks at different language levels are needed to reliably measure the learners’ writing competence 106 3 Methods (Porsch, 2010). The multi-level approach, on the other hand, uses tasks that are able to measure several CEFR language levels, and therefore needs fewer tasks for the assessment (Porsch, 2010). Both approaches fulfil validity and reliability requirements (Porsch & Köller, 2010). Since several studies in Switzerland had shown that by the end of primary school most learners reached level A1 or A2 in EFL writing (see Figure 1.1), the rather narrow range of language levels seemed to allow the use of the multi-level approach. This had the advantage that it was possible to keep the learners’ workload down. Nevertheless, as discussed in chapter 2.3.2, it seemed crucial for at least two tasks from different genres to be used, so that a bias by genre could be avoided and more reliable results achieved. Selecting the genres An analysis of the cantonal curriculum ( BKS , 2018) and the CEFR descriptors for levels A1 and A2 (Szabo, 2018) showed that at these levels mainly descriptive and narrative texts, as well as correspondence, are considered relevant. Even though Hallet (2016) argues that children can already write very simple argumentative texts in the foreign language at primary school level, it was decided not to use this genre, since it was not expected in the curriculum and the children might not have been familiar with it. The three more common genres, descriptive texts, narratives and correspondence, were selected for the study. It was decided that the pupils should write a descriptive text in the form of an e-mail, and a simple story. With these genres, it seemed possible to cover different text functions, while at the same time taking into account the learners’ language level. The task specifications, however, appeared to be of similar importance, since only the concrete realisation of the genres in tasks could show whether it was possible to implement them at a level appropriate for young EFL learners. Selecting the topics Before the actual writing tasks could be specified, it appeared central to select the topics the pupils would have to write about. As discussed in chapter 2.5.3, research in an L1 context showed that young learners’ topic knowledge had an influence on the quality of their narrative, argumentative and informational texts. It therefore seemed important to select topics all learners were familiar with in order to avoid biased results. In addition, it seemed crucial to consider the learners’ language resources (see chapters 2.4.4 and 2.5.1). As Griva and Chostelidou (2013) found, limited language resources were one of the main challenges young EFL learners were concerned with when composing texts in the foreign language. Therefore, it was decided to use topics that were related 107 3.2 Data collection instruments to the learners’ everyday life and that, according to the curriculum ( BKS , 2018), were expected to be dealt with in foreign language classes at primary school. Situating the tasks In addition, it seemed important that the learners could use the foreign lan‐ guage in a communicative and meaningful way (see chapters 2.3.1 and 2.3.3). According to McKay (2006), ‘young learners learn best through activities that are concrete and meaningful, and evidence of their language learning is most likely to be present in language use assessment tasks that have similar characteristics to those in the child’s real world’ (p. 100). Therefore, the two tasks were situated in contexts that tried to highlight their communicative function. The first task (see Figure 3.5) was situated in a (fictional) class collaboration project with a school in Canada. The pupils were asked to write an e-mail and introduce themselves to Sophie and Jacob, two peers from the Canadian school, and find out about them by asking them questions. The second task (see Figure 3.6) was to write a story with a witty ending. Six pictures introduced the learners to the content of the story, namely a fascinating book that kept both child and parent from coming to dinner. Two empty picture frames indicated that the pupils should think about how the story continued and ended. The most difficult aspect of the task design, however, was to find a suitable difficulty level that allowed the learners at level A1.1 to successfully complete the tasks and, at the same time, gave the high-achieving pupils sufficient freedom to display their proficiency. With regard to the e-mail, it was attempted to achieve this by giving instructional scaffolding in the form of aspects that should be covered in the text and by encouraging the learners to add more if they were able to do so: ‘Write about the following aspects and add more if you can’ (see Figure 3.5). Furthermore, in order to elicit more than just a list of facts, the learners were asked to give reasons for their preferences, describe two hobbies in detail and think about what they wanted to know from Sophie and Jacob. In order to inspire the learners, two pictures of Sophie and Jacob were added that not only showed their portraits, but also their hobbies (fishing and photography). In the second task, differentiation appeared to be more difficult, in particular for low level learners. While it seemed possible to write the e-mail with very simple sentences (e. g. Hello Sophie and Jacob! My name is … I’m … years old. I like …), writing a story appeared to be more demanding, in that considerably more creativity and imagination is involved, and the learners may sometimes have more ideas than they are able to express in the foreign language (see the corresponding research findings presented in chapters 2.4.4 and 2.5.1). It was therefore decided to use a picture story that would provide guidance and 108 3 Methods narrow down the topic to world fields which even learners at level A1.1 were expected to be able to cope with, such as family, food, rooms and daily routines. Nevertheless, it also seemed central to open up the task in order to provide high-level learners with the opportunity to be creative, bring in their own ideas and use a wider range of language. This was also important, since the rating criteria not only measured linguistic, but also sociolinguistic and pragmatic aspects (see chapter 3.2.1). Figure 3.5 Writing task 1: E-mail 109 3.2 Data collection instruments Figure 3.6 Writing task 2: Story 110 3 Methods Instructions As discussed in chapter 2.3.3, the instructions also appear to play an impor‐ tant role when young EFL learners are concerned. It seems central for the instructions and pictures to be clear and unambiguous, in order to avoid construct-irrelevant influences on the test results (Cho & So, 2014). Several aspects were considered when formulating the task instructions. First, care was taken to use simple formulations wherever possible. In order to avoid the learners from being hampered in successfully completing the tasks because of comprehension difficulties, a German translation was provided underneath (see Figure 3.5 and Figure 3.6). While this might not be considered good practice in classroom instruction, because the learners are expected to specifically develop the ability to understand instructions in the foreign language ( BKS , 2018), it appeared central, in the context of research, to avoid any construct-irrelevant influence that could have negatively influenced the validity and reliability of the results. Secondly, formulations were avoided which the children could have copied directly from the task sheet into their own texts. Questions such as ‘How old are you? ’ or ‘What are your hobbies? ’, for example, were avoided because the learners could have copied them directly into their own texts as questions to Jacob and Sophie. Thirdly, as discussed in chapter 2.3.1, it appeared important to clearly specify the product goals. In the first task, for example, not only the content was specified, but it was also stated that the learners should give reasons, describe something in detail or ask questions. In the second task, it was emphasised that the story should be coherent (rather than unrelated descriptions of the pictures), detailed enough for the reader to perceive it as vivid and interesting, and written in the past tense. Lastly, as discussed in chapter 2.3.1, it was noted in the test administration script that the instructions should be read aloud to the learners and that they should be given the opportunity to clarify questions. For this purpose, an appendix was added to the test administration script which contained the answers to the most frequently asked questions that had been collected during the piloting phase. External resources and transcription tool Two further questions that had to be addressed were whether external resources such as dictionaries or collaboration with peers should be allowed (see chapter 2.5.4), and what transcription tool should be used (see chapter 2.4.5). Since collaboration was not the primary focus of this research and the aim was to gauge each individual’s writing competence, collaboration was not allowed in this study. With regard to dictionaries, it was also decided not to allow their 111 3.2 Data collection instruments use, since it seemed likely that not all learners were familiar with their use, and therefore would not have had the same conditions for writing the texts. Since research has shown that the quality of texts written by young learners is higher if the texts are written by hand, compared to keyboard (Connelly et al., 2007; Read, 2006), it was decided to have the learners write the texts by hand. Even though it might appear unusual to write an e-mail by hand, it does not seem to be uncommon practice at primary school to first draft a text by hand before it is typed on the keyboard, especially when there are only few computers available in the classroom. Task revision In order to assess the quality of the writing tasks, feedback from experts, teachers, learners and raters was obtained. Initially, the writing tasks were presented to different experts from the fields of primary and secondary school EFL teaching and L1/ EFL writing research. Their feedback led to changes such as a more specific contextualisation of the e-mail or the consideration of testing whether the learners would be able to write the story in the past tense (the first version did not contain this element). In the first piloting phase, the tasks were tested in two classes, and feedback from the teachers and learners was obtained. They gave positive feedback on various aspects such as the timing, the clarity and content of the tasks, and the level of difficulty. One teacher observed that the numbering of the story pictures might tempt the children to write a numbered list of sentences rather than a coherent story. Therefore, the numbers were replaced by arrows in the subsequent version of the writing tasks. Furthermore, the pupils were asked in a questionnaire to assess the comprehensibility of the instructions, the difficulty level of the tasks, and whether they had been given sufficient time for writing the texts (see Table 3.3). 43 pupils completed the questionnaire. One pupil was dyslexic, and three had individual learning objectives in the school language. Item Level of agreement a E-mail Level of agreement a Story n M SD n M SD I have understood what I have to do. 43 3.98 0.15 43 3.86 0.41 I had enough time to write the text. 43 3.93 0.26 43 3.84 0.43 I found it easy to write the text. 43 3.21 0.60 43 2.91 0.81 Note. a As assessed on a 4-point Likert scale from 1 (not true) to 4 (true). Table 3.3 Learner feedback from the piloting (I) 112 3 Methods As can be seen in Table 3.3, the comprehensibility of the instructions and the time allotted for writing were rated very highly. As previously assumed, the e-mail was perceived to be easier (M = 3.21, SD = 0.60) than the story (M = 2.91, SD = 0.81). The pupils were additionally asked whether they found the tasks too difficult (see Table 3.4). For both tasks, more than 90 % of the pupils indicated that they did not find them too difficult (95 % or 41 pupils for the e-mail and 93 % or 40 pupils for the story). Therefore, both tasks appeared to have an appropriate level of difficulty. Level of agreement a Number of pupils n M SD Not true Rather not true Somewhat true True I found the e-mail task too difficult. 43 1.21 0.60 37 4 1 1 I found the story task too difficult. 43 1.26 0.58 35 5 3 0 Note. a As assessed on a 4-point Likert scale from 1 (not true) to 4 (true). Table 3.4 Learner feedback from the piloting (II) After the piloting, the texts were rated by two raters during the rater training. The text rating revealed a high variability of text quality and showed that all learners had been able to complete the two tasks. Some of the learners had written texts that were longer than a DIN A4 page, and therefore a second page for writing was added to the task sheet. Furthermore, the raters observed that some learners wrote very brief e-mails, including only the most necessary information, and that the communicative effect and the learners’ creativity only became apparent in a few learners’ stories. Therefore, the instructions for the e-mail were adapted to include the reasons for the learners’ preferences and the phrase ‘Write more if you can’. Regarding the story, the task of inventing an own, witty story ending was made more explicit, both in the instructions and also visually by adding two empty picture frames. It became apparent that careful consideration needs to be given to the details of the task instructions, in order to ensure that the intended dimensions of writing (see chapter 3.2.1) become visible in the texts. The adapted tasks were once again piloted in the second pilot study that preceded the main data collection. Since the second pilot study was shortened to include only the writing tasks, the learners were asked to give oral rather than written feedback. They reported that they liked the fact that the story had 113 3.2 Data collection instruments an open ending, and again they perceived the e-mail as easier than the story. When asked for feedback on the inclusion of the past tense in the story task, they did not regard it as too difficult or stressful, and therefore it was kept, since it provided valuable information about the complexity of grammatical structures in the texts. 3.2.3 Learner and teacher questionnaires The learner and teacher questionnaires had two purposes. First, they aimed at providing descriptive data about the learners’ and the teachers’ perception of EFL writing and the writing instruction in the classroom. Additionally, they were intended to provide data for different statistical analyses, in particular for identifying and gauging individual and educational factors that were assumed to be predictors of the learners’ EFL writing competence. Learner questionnaire The learner questionnaire (see Appendix B) was provided in paper-pencil format, since the writing tasks were also written by hand, and it was not clear whether all schools would have had enough digital devices to have the learners complete the questionnaire online. The questionnaire consisted of two parts with questions about the learners themselves (e. g. the learners’ self-efficacy, the importance they assigned to learning to write in English, their extra-curricular use of the English language, or what educational track they would be placed in in the future) and questions about classroom instruction (e. g. the frequency of writing English texts in class, the role of pre-writing activities, instructional scaffolding, feedback, vocabulary learning or spelling in class). It was provided in the school language, since a language proficiency at level A1 / A2 would not have been sufficient to complete the questionnaire in English. Before the questionnaire was used in the pilot study, it was tested for compre‐ hensibility and answerability. According to Maiello (2011), comprehensibility and clarity of content are central characteristics of a good questionnaire. He argues that it is no use at all to ask questions that are not understood or are understood in a way other than intended (Maiello, 2011, p. 51). This appears to be of particular importance when working with young learners, who may have a smaller language repertoire than adults and may understand certain terms and questions differently from more experienced learners. Lienert and Raatz (1998) suggest empirically validating the clarity of content of all questionnaire items by testing them with a group of test persons from the target population. A pre-piloting was thus conducted with 21 sixth-grade learners, who assessed 114 3 Methods the clarity of the questionnaire items on a 3-point Likert scale with the response categories completely clear (1), not quite clear (2) and unclear (3). Additionally, they indicated whether they would be able to provide an answer (yes/ no). In order to assess how carefully the learners would read the items, two dummy items were added that contained terms the learners would not understand (Interimsprache and redundant). The data of five learners, who had indicated that these items were completely clear to them, were excluded from the analysis. After the pre-piloting, group interviews were carried out with the learners in order to examine the reasons for the reported incomprehensibility and the learners’ inability to provide answers. Level of comprehensibility a Number of items Percent Level of answerability b Number of items Percent 100.0 55 73.3 100.0 36 48.0 90.0-99.9 15 20.0 90.0-99.9 23 30.7 80.0-89.9 4 5.3 80.0-89.9 14 18.7 < 80.0 1 1.3 < 80.0 2 2.7 Note. a Percentage of learners who assessed an item as completely comprehensible. b Percentage of learners who assessed an item as answerable. Table 3.5 Comprehensibility and answerability of the original questionnaire items As can be seen in Table 3.5, a large number of questionnaire items showed very high levels of comprehensibility and answerability. 73 % of the questionnaire items were rated as completely comprehensible by all learners. Only five items had a level of comprehensibility below 90 %, which means that two or three learners had indicated that the item was not fully clear to them. The level of answerability was also high, but lower than the level of comprehensibility. 48 % of the questionnaire items were rated as answerable by all learners, and 21 % of the items had a level of answerability below 90 %, with two, three or once five learners indicating that they were not able to give an answer. The subsequent group interviews provided useful information on how the items with low comprehensibility and answerability could be improved. Table 3.6 shows some examples of how the questionnaire items were improved. 115 3.2 Data collection instruments Original item Feedback Adaptation Wie häufig schreibt ihr […] kurze Abschnitte (2-4 zu‐ sammenhängende Sätze)? How often do you write […] short paragraphs (2-4 con‐ nected sentences)? Unclear what the word zusammenhän‐ gend (connected) means. Wie häufig schreibt ihr […] kurze Abschnitte (2-4 Sätze)? How often do you write […] short paragraphs (2-4 sen‐ tences)? Wie oft sprecht ihr bei ge‐ meinsamen Aktivitäten Eng‐ lisch (z. B. Spiele, Lieder, Ge‐ schichten vorlesen, Ausflüge, Ferien)? How often do you speak Eng‐ lish during joint activities (e. g. games, songs, reading stories, excursions, holidays)? I did not fully under‐ stand the word Aktivi‐ täten (activities). Wie oft sprecht ihr Englisch, wenn ihr gemeinsam etwas macht (z. B. Spiele, Lieder, Ge‐ schichten vorlesen, Ausflüge, Ferien)? How often do you speak Eng‐ lish when you do things to‐ gether (e. g. games, songs, reading stories, excursions, holidays)? Während dem Schreiben dürfen wir Wörter nach‐ schauen (z. B. in einem Wör‐ terbuch). While writing, we are allowed to look up words (e. g. in a dictionary). We are allowed to look up words in the course book. It would be good to add in the course book. Während dem Schreiben dürfen wir Wörter nach‐ schauen (z. B. in einem Wör‐ terbuch oder im Englisch‐ buch). While writing we are allowed to look up words (e. g. in a dic‐ tionary or in the course book). Table 3.6 Examples of how the questionnaire items were improved For information on scales, indices and reliability analyses, see chapter 3.4.4 and the scale documentation in Appendix C. Teacher questionnaire In contrast to the learner questionnaire, the questionnaire for the teachers was provided online. It contained questions about the teachers themselves (e. g. their teaching experience, teaching diplomas and language level), about their class (e. g. the number of pupils in the class and the ID numbers of learners with special pre-conditions such as dyslexia or individual learning objectives) and about their classroom instruction (the same questions as in the learner questionnaire and some additional items such as the aims the teachers pursue when teaching EFL writing). In addition, the teachers were asked to estimate the learners’ EFL writing competence and to provide the learners’ overall mark in the school subject English so that the criterion validity of the learners’ writing scores could be calculated (see chapter 3.4.3). The scale 116 3 Methods for estimating the learners’ EFL writing competence was formulated based on relevant CEFR descriptors (Council of Europe, 2001; Szabo, 2016) and descriptors from the curriculum of the Canton of Aargau ( BKS , 2014; D- EDK , 2015), see Table 3.7. Since school marks are highly sensitive data, the teachers were explicitly informed about the purpose of the request for this information, and confidentiality was assured. Language level Descriptor Above level A2.1 The learners can write a slightly longer, coherent text in English. They already use more difficult words and language structures. Some simple structures are used correctly and short, common words are written reasonably correctly. Level A2.1 The learners can write a simple text in English. The text contains mainly simple, but also some more difficult words and language structures. Some simple structures and short, common words are used and written reasonably correctly, but still with errors. Level A1.2 The learners can write a short, simple text in English. They use simple words and language structures. These are not yet correct, but are generally comprehensible. Level A1.1 The learners can write down simple information in English using short sentences. They use very simple words and language struc‐ tures. Starter phase The learners can write short, at times incomplete sentences in English. Table 3.7 Scale for the teachers to judge the learners’ EFL writing competence (transla‐ tion) For information on scales and indices in the teacher questionnaire, as well as re‐ liability analyses, see chapter 3.4.4 and the scale documentation in Appendix C. 3.2.4 Learner interviews The learner interviews were intended to provide additional insights into the learners’ perception of EFL writing at primary school, and thus to allow for data triangulation. The interviews focused on the learners’ perception of the writing process, the types of texts they had already composed, the writing instruction they experienced in class and their extra-curricular use of the English language. It was decided to conduct a semi-structured interview, which gives guidance in the form of interview guidelines, but at the same time provides 117 3.2 Data collection instruments the opportunity to give room to and expand on what the children say (Daase, Hinrichs, & Settinieri, 2014). The interview guidelines are available from the author on request. 3.3 Sampling method and participants In order to recruit participants for the study, all primary schools in the Canton of Aargau were contacted, whereupon 47 sixth grade classes applied to participate. From these classes 19 were selected for participation, using a stratified random sampling method (Bortz & Döring, 2006) that grouped the classes according to the frequency of their writing English texts in class. Since most teachers had indicated a moderate frequency of writing texts in class and only few a low or high frequency, this sampling method was selected to avoid the final selection of classes consisting only of classes with a moderate frequency of writing English texts. It ensured that the final sample had the same proportion of classes with low, moderate and high writing frequency as the total number of classes that had applied for participation. Nevertheless, as the sampling is based on voluntary participation, a certain bias cannot be completely precluded, and conclusions about the population must be drawn with some caution. The sample consisted of 19 classes from 19 different schools with a total number of 332 pupils who had been given permission to participate. 10 pupils were ill on the day of data collection, which resulted in a final sample of n = 322 pupils. The schools were geographically spread across all eleven districts of the Canton of Aargau and located in both urban and rural areas. 13 of the classes were regular classes, two were part of a mixed-age classroom setting, and at two schools English was taught in groups that were made up of pupils from different sixth grade classes. The number of pupils per class ranged between 10 and 29, with a mean of 19 pupils. All 19 teachers worked as subject teachers for English, and their professional experience as primary school EFL teachers ranged between 1 and 12 years, with a mean of 7.3 years. 12 teachers (63 %) held an official teaching diploma for teaching English at primary school, four teachers (21 %) had an English language teaching diploma that was similar to the required one, or one for a different target age group (e. g. for teaching English at secondary schools or to adults), and three teachers (16 %) had no diploma for teaching English as a foreign language. 15 teachers (79 %) had an English language proficiency at CEFR level C1 or C2, three teachers (16 %) were native English speakers and one teacher did not know her language level. 118 3 Methods The learners were all in grade 6 and approximately 12-13 years old. 52 % were female and 48 % male learners. For the great majority of pupils (88 %) it was their fourth year of learning English. 3 % had less than four years’ experience of learning the English language, and 9 % had more than four years. There were seven pupils (2 %) who were native English speakers or had close contact to the English language, e. g. through attending an English-speaking school. 83 learners (26 %) had an L1 other than German, and 20 learners (6 %) had some sort of learning difficulty. Nine learners additionally participated in a learner interview. They were selected from three classes who showed a low, medium and high average writing proficiency. From each of the three classes, three learners were selected who had shown a low, medium and high performance on the writing tasks. 3.4 Data collection and processing 3.4.1 Ethical considerations Various measures were adopted to comply with research ethics. All parties involved were informed about the content and aims of the study, the data collection procedure and about how anonymity and confidentiality would be ensured. Informed consent was obtained from the cantonal department of education, the school principals and the parents. The Department of Education and all school principals gave their consent, and of the 362 pupils, 332 (91.7 %) were given permission to participate in the study. Additional consent was obtained from the school principals and parents for the learner interviews. In order to ensure anonymity, all pupils were given a personal ID number, which was assigned by the teachers so that only they, but not the researchers, knew the learners’ identity. The writing tasks and questionnaires were piloted to ensure that the comprehensibility, difficulty level and time specifications were adequate, and that the demands placed on the pupils were not too high. Feedback from both learners and teachers was obtained in order to identify any difficulties or problems that might have occurred. To ensure transparency, the test administrators informed the pupils about the data collection procedure, the different tasks they would complete and the criteria that would be used to assess their text. Three months after the completion of the main data collection, the teachers and learners received feedback on the learners’ writing competence. The feedback to the pupils showed the different qualities of the two texts and the competency level the learners had achieved. The competency levels had been 119 3.4 Data collection and processing formulated based on the curriculum of the Canton of Aargau ( BKS , 2014; D- EDK , 2015) and relevant CEFR descriptors (Council of Europe, 2001; Szabo, 2016). Since the standard setting (see chapter 3.5) had not yet taken place at that time, provisional cut scores were used that were obtained from a small, task-specific standard setting with only two panellists. A scale bar on the feedback sheet indicated whether the competency level had been clearly reached or whether the text was on the border between two competency levels. The teachers received a more detailed feedback with the results of the whole class. 3.4.2 Data collection Prior to the main data collection, a pre-piloting of the learner questionnaire (see chapter 3.2.3) and two pilot studies (see chapters 3.2.2 and 3.2.3) were implemented. The data and text products from these pilot studies were used to validate and revise the learner questionnaire and writing tasks, to train the raters and to refine the rating scales and the data collection procedure. The main data collection took place from January to March 2018, about 4-6 months before the end of the school year. It was implemented by three well instructed test administrators, who followed a test administration script with clear specifications of the test procedure, the concrete wording of the instructions and sample answers for frequently asked questions. During data collection, the teacher was present in the classroom and supported the test administrator in distributing, collecting and checking the documents. Each part of the data collection (the two writing tasks and the questionnaire) lasted 20 minutes, with five-minute breaks in between. Learners who finished early were provided with an activity sheet to bridge the time until the next task started. The test administrators kept a protocol and asked for a short feedback from the teacher. The main data collection was followed by the online teacher questionnaire (see chapter 3.2.3), which was provided towards the end of June 2018. Since the questionnaire also required the teachers to assess the learners' writing competence for validation purposes, it was provided before the teachers received the feedback on how well their classes and the individual pupils had performed on the writing tasks. At about the same time, the semi-structured interviews with the nine learners from three different classes were carried out. With the consent of the pupils and the parents, the interviews were recorded and later transcribed (see chapter 3.4.1). 120 3 Methods 3.4.3 Rating procedure, summary scoring, Rasch analysis and criterion validity of the writing scores Rating procedure The 322 learners composed a total of 644 texts. These texts were first transcribed into a digital format in order to avoid the rating being influenced by the learners’ handwriting, and in order to simplify the rating process. Spelling errors and any other errors were not corrected. After the transcription, all texts were checked by another person for transcription errors. Two raters, who were both experienced primary school teachers with a good English language proficiency, rated the texts independently using the rating scales, the support documents for cases of uncertainty and the benchmark texts as presented in chapter 3.2.1. 337 texts (175 e-mails and 162 stories) were rated by both raters, and 307 texts were rated by only one rater. Inter-rater reliability was regularly monitored. Intraclass correlation coefficient ( ICC ) estimates and their 95 % confident intervals were calculated using IBM SPSS Statistics (Version 25) based on a two-way mixed-effects model with a single measure and absolute agreement. The raters reached an inter-rater reliability between 0.78 and 0.94 for the different dimensions of text quality (see Table 3.8), which can be regarded as a good reliability (Koo & Li, 2016). Intraclass correlation a 95 % Confidence interval Lower bound Upper bound E-mail (n = 175) Coverage 0.92 0.89 0.94 Level of detail 0.86 0.81 0.89 Genre-specific elements of an e-mail 0.94 0.92 0.96 Coherence 0.82 0.75 0.86 Cohesion 0.86 0.80 0.90 Complexity of syntax and grammar 0.82 0.76 0.86 Correctness of syntax and grammar 0.78 0.70 0.84 Vocabulary 0.84 0.79 0.88 121 3.4 Data collection and processing Orthography 0.85 0.81 0.89 Punctuation 0.88 0.85 0.91 Story (n = 162) Communicative effect / crea‐ tivity 0.82 0.76 0.86 Level of detail 0.88 0.84 0.91 Coherence 0.81 0.74 0.85 Cohesion 0.84 0.79 0.88 Complexity of syntax and grammar 0.84 0.75 0.89 Correctness of syntax and grammar 0.84 0.78 0.88 Vocabulary 0.91 0.88 0.94 Orthography 0.86 0.81 0.89 Punctuation 0.90 0.86 0.92 Note. a ICC, two-way mixed, single measure, absolute agreement. Table 3.8 Inter-rater reliability of the text ratings Summary scoring In order to calculate the final writing scores, the following procedure was applied: First, the averages of the scores from the two raters were calculated (where applicable); then, in order to allow for a balanced weighting, the averages of the different subcategories from the corresponding dimensions of text quality (see Table 3.9). As a result, the subcategories, which comprised of one, two or three dimensions of text quality, were given equal weighting. For genre-specific analyses, the averages of these subcategories were used. In order to calculate an overall writing score, a many-facet Rasch analysis was conducted that adjusted the scores for task difficulty, rater severity and difficulty of the rating criteria (see next section). 122 3 Methods Subcategories Dimensions of text quality E-mail Task completion Coverage, level of detail, genre-specific elements of an e-mail Text structure and cohesion Coherence, cohesion Syntax and grammar Complexity and correctness of syntax and grammar Vocabulary Vocabulary range Language mechanics Orthography, punctuation Story Task completion Communicative effect / creativity, level of detail Text structure and cohesion Coherence, cohesion Syntax and grammar Complexity and correctness of syntax and grammar Vocabulary Vocabulary range Language mechanics Orthography, punctuation Table 3.9 Subcategories used for calculating the final writing scores (e-mail and story) Many-facet Rasch analysis In order to adjust the final writing scores for task difficulty, rater severity and difficulty of the rating criteria, a many-facet Rasch analysis was run using the software FACETS (Version 3.80.4; Linacre, 2018). A four-facet rating scale model was devised that encompassed the four facets examinee writing proficiency, task difficulty, rater severity and criteria difficulty, and used a weighting of the criteria as presented in Table 3.9. The global model fit was assessed by analysing the unexpected responses with z standardised residuals. According to Linacre (2018), the data fit the model when about 5 % of the absolute standardised residuals are equal to or greater than 2, and about 1 % is equal to or greater than 3 (p. 167). The analysis showed that 4.42 % of the responses had an absolute standardised residual equal or greater than 2 and 0.51 % an absolute standardised residual equal or greater than 3. These figures, thus, appear to indicate a satisfactory model fit. Figure 3.7 shows the Wright map that illustrates the results of the many-facet Rasch analysis. The four facets examinee writing proficiency, task difficulty, rater severity and criteria difficulty were calibrated onto the same logit scale. 123 3.4 Data collection and processing Note. A star in the examinee column represents five pupils and a dot one pupil. Figure 3.7 Wright map illustrating the four-facet Rasch analysis The stars and dots in the examinee column represent the learners’ writing proficiency with high proficiencies appearing at the top and low proficiencies appearing at the bottom of the column. The task column shows that the story was slightly more difficult than the e-mail. The rater column compares the two raters in terms of rater severity. According to the Wright map, the two raters were equally severe in their ratings. The criteria column displays the difficulty of obtaining a certain rating for the different rating criteria. Coverage (e-mail) was the easiest criterion, while macro dimensions such as coherence, level of detail, genre-specific elements (e-mail) and communicative effect / creativity (story) were 124 3 Methods the most difficult criteria. The last column calibrated the five-category rating scale onto the same logit scale. Taking task difficulty, rater severity and criteria difficulty into account resulted in only small adjustments of the learners’ final scores. The absolute adjustment of the raw scores ranged between 0.00 and 0.05. Only seven learners (2 %) would have been assigned to a different competence level if the scores had not been adjusted. Table 3.10 presents six sample records from the Rasch analysis. The learners C and E are two of the seven learners whose language level changed because of the Rasch score adjustment. Examinee Logit a SE (Raw) Score Language level Observed Adjusted Based on observed scores Based on adjusted scores A -1.18 0.22 1.39 1.40 A1.1 A1.1 B -0.42 0.15 1.70 1.70 A1.2 A1.2 C 0.99 0.14 2.32 2.33 A1.2 A2.1 D 1.89 0.19 2.74 2.77 A2.1 A2.1 E 3.01 0.20 3.26 3.31 A2.1 A2.2 F 3.97 0.18 3.62 3.65 A2.2 A2.2 Note. a Logit (log-odds unit) value, as presented in the Wright map in Figure 3.7. Table 3.10 Six sample records from the many-facet Rasch analysis In summary, the results of the many-facet Rasch analysis appear to confirm the high reliability of the ratings. The two raters were similarly strict in their assessment, and only minor adjustments were made to the scores. Criterion validity of the Rasch-adjusted writing scores In order to assess the criterion validity of the writing scores (which compares them with a valid external criterion, see Grotjahn & Kleppin, 2017b, pp. 56-57), the scores were compared to the teachers’ estimation of the learners’ EFL writing competence and the learners’ overall mark for the school subject English. Before the teachers received the results of their classes, they completed a questionnaire that asked them to assess their learners’ writing competence and to provide the learners’ marks for the school subject English (see chapter 3.2.3). 125 3.4 Data collection and processing A Spearman’s rank-order correlation was run to assess the relationship between the learners’ writing scores and the teachers’ estimation of the learners’ EFL writing competence, as well as the learners’ mark for the school subject English. A preliminary analysis showed that the relationships between the variables were monotonic, as assessed by visual inspection of scatterplots. There was a statistically significant, strong positive correlation between the learners’ writing scores and the teachers’ estimation of the learners’ EFL writing competence, r s (320) = 0.70, p < .001, which is a large effect size according to Cohen (1988); and a slightly lower but still statistically significant, strong positive correlation between the learners’ writing scores and their mark for the school subject English, r s (319) = 0.67, p < .001, which is a large effect size according to Cohen (1988). Compared to Südkamp, Kaiser, and Möller’s meta-analysis (2012), which in‐ vestigated the correspondence between teachers’ judgments and their learners’ academic achievement and revealed an overall average effect size of ES = 0.63 across the 75 analysed studies, the correlations (or effect sizes) of r = 0.70 and r = 0.67 in this study appear to indicate a good criterion validity of the writing scores. For informed teacher judgements, where the teachers were informed about the standard of comparison, however, Südkamp et al. (2012) found an overall average effect size of ES = 0.76. Since the teachers in this study were informed about the standard of comparison when estimating the learners’ EFL writing competence (see Table 3.7), the correlation or effect size of r = 0.70 must be considered to be slightly lower than the observed average, but might still be considered as satisfactory. 3.4.4 Processing of questionnaire data: scales, indices and reliability analyses Since the learner questionnaire was provided on paper, the data were first digitalised and then carefully checked for data entry errors. The teacher ques‐ tionnaire was provided online. The data from both questionnaires were prepared for data analysis, evaluated with regard to their quality and summarised in a codebook. The following section presents the different scales and indices used for statistical analyses, and the corresponding reliability analyses. The term scale is used to refer to a set of questionnaire items whose individual elements are intended to measure the same theoretical concept or dimension (Latcheva & Davidov, 2019, p. 895). The term index, on the other hand, refers to a set of questionnaire items whose individual elements measure different dimensions of a construct and are combined into a new variable (Latcheva & Davidov, 2019, 126 3 Methods p. 895). While for a scale a correlation of the individual questionnaire items is expected, because they describe the same dimension of a latent construct, the individual items of an index do not necessarily have to correlate with each other, because they cover different dimensions of a construct (Latcheva & Davidov, 2019). In the scale and index documentation that accompanies the following section (see Appendix C), correlations and alpha coefficients, which describe the internal consistency of a set of questionnaire items, are presented for both, scales and indices, for reasons of comparability, even if they might not be required for indices. Caution, however, seems to be required with regard to correlations and alpha coefficients for the scales and indices from the teacher questionnaire, as the sample size is very small (n = 19), and outliers might have a considerable influence. All scales and indices were formed by calculating the unweighted average of the different items. The learners’ self-efficacy in writing in English As discussed in chapter 2.7.2, self-efficacy is defined as ‘an individual’s assess‐ ment of his or her competence to perform a future task’ (Troia et al., 2013, p. 18). Since no self-efficacy scale for EFL writing in a young learner context appeared to exist, it had to be developed on the basis of scales from different contexts. From a self-efficacy scale for L1 writing by Shell, Colvin, and Bruning (1995), one item was used and translated into German: Ich weiss, dass ich auf Englisch so schreiben kann, dass der Leser es versteht (from ‘get your point across in your writing’, p. 388). The other items were task-specific or focused primarily on accuracy, e. g. ‘correctly punctuate a sentence’ or ‘use correct plurals, prefixes, and suffixes in your writing’ (Shell et al., 1995, p. 388), and thus could not be considered or seemed to have a too narrow focus to adequately represent the definition of EFL writing as presented in chapter 2.1. From a self-efficacy scale for learning English by Wagner, Helmke, and Rösner (2009, p. 59), three further items were selected and adapted to EFL writing: Ich weiss, dass ich gute englische Sätze schreiben kann (I know that I can write good English sentences); Wenn ich mir beim Schreiben auf Englisch Mühe gebe, dann kann ich es auch (If I make an effort when writing in English, I am successful); and Ich arbeite auch dann weiter, wenn das Schreiben auf Englisch mühsam ist (I continue to work even when writing in English is troublesome). The reliability of the self-efficacy scale for writing in English was assessed by calculating Cronbach’s alpha. The scale consisted of the four items presented above and had an internal consistency of α = 0.72. The analysis showed that the reliability could be improved to α = 0.74 if one item was deleted. The item was 127 3.4 Data collection and processing thus removed from the scale in order to reach the highest possible consistency (see Appendix C, Table C1 for the details). The learners’ self-efficacy in learning English In order to measure the learners’ self-efficacy in learning English, a scale from Wagner et al. (2009) was adapted to match the slightly younger target age group in this study (grade 6 instead of grade 9). In particular, the formulation of the different items was simplified to ensure that the pupils would understand the items (see chapter 3.2.3 for how the comprehensibility of the items was tested). The scale consisted of five items (see Appendix C, Table C2), and its reliability was assessed by calculating Cronbach’s alpha. The scale had an internal consistency of α = 0.77. The analysis showed that the reliability could be improved to α = 0.79 if one item was deleted. The item was thus removed from the scale in order to reach the highest possible consistency (see Appendix C, Table C2). The learners’ extra-curricular use of English As described in chapters 2.9.2 and 2.10.3, the study also aimed at investigating the influence of the learners’ extra-curricular use of English on their EFL writing competence. The index for measuring the learners’ extra-curricular use of English was adapted from Wagner et al. (2009, p. 39) and extended with items from Sylvén and Sundqvist (2012, p. 321). The index consisted of eight items, such as the frequency of watching TV , movies or videos in English; of listening to English music or audio books; of speaking, reading or writing in English; or of being in contact with English-speaking people (see Appendix C, Table C3, for the complete list of items). The item analysis (see Appendix C, Table C3) showed that the item Wie oft hörst du dir englische Musik oder Hörbücher an? (How often do you listen to English music or audio books? ) had a corrected item-total correlation of only r it = 0.19 and thus seemed to be measuring a clearly distinct dimension of the learners’ extra-curricular use of English. The item had a higher mean than all other items, indicating that listening to English music or audio books was more frequent than any of the other activities. 62 % of the learners stated that they listened to English music or audio books every day, and 24 % once or twice a week. Despite this clearly distinct item, the index had a high internal consistency of α = 0.79 (see Appendix C, Table C3). 128 3 Methods Classroom instruction In order to investigate the influence of different classroom variables on the learners’ EFL writing competence (see research question III .3 in chapter 2.10.3), the learners and teachers were asked to indicate whether or how frequently different aspects of classroom instruction were applied in class. The constructs that were operationalised were instructional elements related to the process and resource level of the model of writing competence for young EFL learners, and to the learners’ familiarity with EFL writing (see Figure 3.8). Figure 3.8 Elements of classroom instruction that were surveyed in the learner and teacher questionnaires A descriptive analysis of the learners’ responses revealed a large inconsistency within the classes. When asked, for example, how frequently text composition was practised in class in grade 6 (see Table 3.11, variables 9.3 and 9.4), the learners’ responses in more than half of the classes ranged from never to once or twice a week. A similar disagreement within the classes was also found for the other elements of classroom instruction. One reason for this inconsistency may be that the learners might not be able to overview a whole school year and reliably remember what was done in class throughout the year. Another reason could be that the learners’ perception of classroom practices is highly subjective, with different learners paying attention to different aspects. As a consequence, the learners’ responses to these aspects had to be considered as lacking the necessary reliability for further statistical analyses. The teachers’ responses to these questions, however, were considered more reliable, since they are usually required to keep the teaching plan for the whole year in mind when planning their lessons, and thus may be more aware of the classroom practices throughout a whole year. 129 3.4 Data collection and processing Wie häufig schreibt ihr in der 6. Klasse im Englischunterricht oder als Eng‐ lisch-Hausaufgabe … How often do you write in the 6 th -grade English classes or as English home‐ work … v_9.1 … einzelne Wörter und Sätze? … single words and sentences? v_9.2 … kurze Abschnitte (2-4 Sätze)? … short paragraphs (2-4 sentences)? v_9.3 … kurze Texte? … short texts? v_9.4 … längere Texte (eine halbe A4 Seite oder mehr)? … longer texts (half an A4 page or more)? Table 3.11 Questionnaire items asking the learners about the frequency of writing in class or as homework The questionnaire items for measuring classroom practices were specifically developed for this project and are related to the theory and empirical findings presented in chapter 2. Classroom practices were selected that appeared to be of particular relevance or interest in the context of teaching EFL writing to young learners, namely different types of support during the writing process and the role of language resources (see chapters 2.4 and 2.5). Appendix C presents the different scales, indices and questionnaire items that were used for measuring the different educational factors in the teacher questionnaire (see Table C4 to Table C10). The correlations and alpha coefficients for these scales and indices should be treated with caution, since the sample size was very small (n = 19), and most constructs were operationalised in the form of indices that measured different dimensions of the latent construct rather than one single dimension. 3.4.5 Interview transcription The learner interviews were conducted in German, audio recorded and tran‐ scribed following a previously defined transcription key. In order to check the transcription accuracy, a second person reviewed all transcripts and made changes where necessary. 3.5 Standard setting This chapter gives account of a standard setting that was implemented in order to align the learners’ writing scores to the CEFR language levels. A standard setting is defined as ‘the process of establishing one or more cut scores on examinations. The cut scores divide the distribution of examinees’ 130 3 Methods test performances into two or more categories’ (Cizek & Bunch, 2007, p. 5), in this case, the CEFR language levels. The purpose of this standard setting was to identify what CEFR language levels the learners reached in EFL writing in order to allow for further analyses that were based on the CEFR levels. The study, for example, tried to identify the percentage of learners that reached the minimum requirements for EFL writing as stated in the Swiss national standards for foreign language learning ( EDK , 2011) and in the cantonal curriculum ( BKS , 2018) ( RQ I.1), to examine what language level could be expected of the learners when they entered the different educational tracks at secondary school ( RQ I.2), and to analyse the characteristics and qualities of young EFL learners’ texts at different language levels ( RQ I.3- RQ I.5). This chapter first describes the selection of the standard setting method. This is followed by a description of the first implementation of the standard setting, which followed Body of Work Method (Cizek & Bunch, 2007; Kingston & Tiemann, 2012), of its challenges and unexpected results. The chapter analyses and discusses possible causes for these results and describes the measures taken as a consequence. This is followed by a description of the second, adapted implementation of the standard setting, and a presentation of the final results. 3.5.1 Selection of the standard setting method It appears crucial to note that a standard setting cannot claim to be flawless since it is, by its nature, a judgement-based decision made by experts and thus necessarily subjective (Cizek & Bunch, 2007). Various standard setting methods have been developed that aim at facilitating this judgement (Cizek, 2012; Cizek & Bunch, 2007). They ‘involve, to one degree or another, human beings expressing informed judgments based on the best evidence available to them, and these judgments are summarized in some systematic way, typically with the aid of a mathematical model, to produce one or more cut scores’ (Cizek & Bunch, 2007, p. 65). According to Cizek and Bunch (2007), several aspects need to be considered when selecting a standard setting method, namely the purpose of the test, the complexity of the knowledge, skills or abilities to be assessed, the test format, the number of performance categories and the availability of resources such as time or money. After a detailed examination of various methods, four were shortlisted, namely the Booklet Classification Method (Loomis & Bourque, 2001), the Body of Work Method (Cizek & Bunch, 2007; Kingston & Tiemann, 2012), the Performance Profile Method (Tannenbaum & Baron, 2010; Zieky, Perie, 131 3.5 Standard setting & Livingston, 2008) and the Examinee Paper Selection Method (Hambleton, Jaeger, Plake, & Mills, 2000). These methods appeared suitable for use with complex multidimensional constructs such as EFL writing that are assessed with so-called constructed response items that ‘ask students to apply knowledge, skills, and critical thinking abilities to real-world, standards-driven performance tasks’ (Tankersley, 2007, p. 11). Since the Body of Work Method is considered ‘perhaps the most widely used of the holistic methods’ (Cizek & Bunch, 2007, p. 117), and because it is well documented and researched (see e. g. Cizek & Bunch, 2007; Kingston & Tiemann, 2012), it was selected to be used in this study. According to Kingston and Tiemann (2012), participants like it because it ‘is similar to other experiences that educators (and others) do on a regular basis’ (p. 215), standard errors are small, and the results are stable when the standard setting is repeated. 3.5.2 Implementation of the Body of Work standard setting method Performance level descriptors One of the main challenges of the standard setting was the fact that the Swiss national standards for foreign language learning ( EDK , 2011) and the cantonal curriculum ( BKS , 2018) not only distinguish between the levels A1 and A2, but further specify the sub-levels A1.1 / A1.2 and A2.1 / A2.2. The minimum standard for writing at the end of primary school is level A1.2 ( BKS , 2018; EDK , 2011). Therefore, it was necessary not only to establish cut scores between the levels A1 and A2, but also between the corresponding sub-levels. In particular, the cut score between the levels A1.1 and A1.2 appeared crucial since it would define whether or not a learner met the minimum requirements of the curriculum. While many CEFR scales in Szabo (2018) distinguish between the levels A2 and A2+ (A2.2), no such distinction is made between the levels A1.1 and A1.2. Thus, further related documents such as Lingualevel ( BKZ et al., 2008) and the cantonal curriculum ( BKS , 2018) were consulted to analyse how they distinguished between these levels. For level A1.1, there were only three descriptors in each document. Some of these descriptors focused on writing at word level (e. g. filling in forms, labelling pictures or creating wordlists), while the remaining did not seem to be clearly distinct from those categorised as level pre-A1.1 in Szabo (2018), probably because Lingualevel ( BKZ et al., 2008) as well as the cantonal curriculum ( BKS , 2018) do not distinguish between pre-A1 and A1.1. Thus it appeared reasonable to provide the standard setting participants with the official CEFR descriptors only and ask them to divide the texts at level A1 into two groups representing a lower and a higher A1 level respectively. 132 3 Methods In the Lingualevel assessment grid for writing ( BKZ et al., 2008), however, a scale for vocabulary was found that seemed to accord well with the CEFR scale for vocabulary range and to distinguish clearly between the levels A1.1 and A1.2. Therefore, this scale was added to the CEFR descriptors used for the standard setting in order to give the participants an example of how the two levels could be differentiated. Further text quality scales that helped to support this distinction could not be identified because they were either not specifically linked to the CEFR language levels (e. g. the descriptors for language resources in the curriculum), focused on different types of tasks (e. g. writing at word level or copying), or because their formulation appeared to be unclear. The final performance level descriptors (available from the author on request) were selected based on their relevance with regard to the writing tasks used in the study (creative writing, correspondence) and with the intention of representing the different pragmatic (coherence and cohesion), sociolinguistic (sociolinguistic appropriateness) and linguistic qualities of the texts (vocabulary range, grammatical accuracy and orthographic control). Selecting the profiles A learner’s profile consisted of the two texts he or she had written (see chapter 3.2.2 for the two tasks and chapter 4.1.3 for some sample profiles). Profiles were selected for the standard setting that fulfilled the following three criteria: They had been rated by two raters; the final scores awarded by the two raters differed by 0.3 or less; and the difference between the two texts was 0.5 or less. These criteria were applied to simplify the standard setting task for the panellists and to reduce the time needed for analysing and discussing the texts. 102 texts fulfilled the criteria. Seven further profiles were excluded because of problematic content such as violence against family members, which might have distracted the panellists from the actual task. The remaining 95 texts provided the pool of profiles that were used for the standard setting. Selecting and training the panellists Ten panellists agreed to participate in the standard setting, two of whom were unfortunately unable to attend at short notice. Thus, the standard setting was conducted with eight panellists. The panellists were selected based on their experience in the field of English language teaching. Seven of them had been or were currently working as EFL teachers at primary school. Two were professors for English language teaching, one was a lecturer, two were teacher trainers who ran professional development courses for primary school EFL teachers, and three were highly experienced EFL teachers. Six panellists were very familiar 133 3.5 Standard setting with the CEFR and had already used it for either practical, academic, scientific or assessment purposes. Two panellists were roughly familiar with it. Different measures were implemented in order to ensure that the panellists had a clear understanding of the CEFR level descriptors, the standard setting procedure and the task of aligning learner profiles to the CEFR language levels. Prior to the event, the panellists were informed about the purpose and programme of the standard setting and provided with the CEFR performance level descriptors, the two writing tasks and two sample profiles. They were asked to study the documents and to highlight those elements in the CEFR performance level descriptors that would help to distinguish the different language levels from each other. At the standard setting, its purpose and procedure were explained, followed by a jigsaw activity where the panellists had to reassemble the CEFR level descriptors that had been cut apart. The aim of this activity was that the panellists would read the individual descriptors carefully and deliberately engage with them so that the descriptors could be recalled to the memory. After this activity, the panellists discussed the CEFR language levels in detail and took notes of the particular features that distinguished the different levels, first in groups and then in plenum. In particular, they were requested to discuss the distinction between the lower and higher A1 level (A1.1 and A1.2). In a survey that was intended to evaluate the standard setting process, all panellists indicated that this introduction had provided them with a clear understanding of the purpose of the meeting, and that the activities and discussions had helped them to develop a clear understanding of the CEFR levels. Procedure The procedure of the standard setting followed the Body of Work standard setting method, which was slightly adapted to the particular context. It is a holistic method where the panellists review a comparably large number of profiles fairly quickly to assign them to one of the performance levels (Cizek & Bunch, 2007). The profiles are presented in folders or booklets, ordered by their final scores. The panellists do not know the scores of the profiles, but they are aware of their ascending order (Cizek & Bunch, 2007). The procedure of the Body of Work Method consists of two rounds. The first round is called range finding and ‘serves primarily as a means of rough identification of cut scores’ (Cizek & Bunch, 2007, p. 127). The panellists read the profiles, which range from the lowest to the highest scores, and assign each to one of the performance levels. This data is used to determine the approximate location of the cut scores. After a break, the panellists are provided with profiles whose scores are located around the preliminary cut scores from round one. In this second round, called 134 3 Methods pinpointing, the panellists have the same task as in the first round, namely to assign each profile to one of the performance levels (Cizek & Bunch, 2007). Based on this data, final cut scores are calculated with a mathematical procedure that is based on logistic regression (see Cizek & Bunch, 2007 for a detailed description of this procedure). For round one, all panellists received a booklet with 16 profiles that ranged from the lowest to the highest scores. They were given one hour to align the profiles to the CEFR language levels. The amount of time needed for this task was estimated based on the experience from a trial run. The panellists were asked not to engage in any discussions while reading the profiles, but to make a note of any questions or uncertainties, so that they would not disturb each other. They were instructed to assess the texts holistically rather than making ratings of the individual dimensions and adding them up. When they had finished, the panellists discussed their ratings in small groups and, if necessary, adjusted them based on new insights from the discussions. At lunchtime, provisional cut scores were calculated and new booklets for the second round printed out that contained eight profiles around each cut score. In the afternoon, the panellists aligned a total of 40 profiles (eight around each cut score) to the CEFR language levels. Again, they discussed their ratings in small groups, made adjustments where necessary and entered their final ratings into an online Excel sheet that calculated the final cut scores. In order to assess the procedural validity of the standard setting (Tannenbaum & Cho, 2014), the panellists completed a survey that required them to evaluate the different steps and components of the standard setting process. Table 3.12 presents the results of this survey. It shows that the panellists regarded all aspects as adequately implemented. Standard setting component Level of agreement a n M SD The workshop leader clearly explained how to perform the task. 8 4.00 0.00 The material provided (writing tasks / CEFR grid / text book‐ lets) supported the process of classifying the student profiles. 8 3.88 0.35 The time provided for the classification of the student profiles was sufficient. 8 3.88 0.35 The time provided for the group and plenum discussions was adequate. 8 3.63 0.52 135 3.5 Standard setting Standard setting component Level of agreement a n M SD The group discussions supported the process of classifying the student profiles. 8 3.75 0.46 The facilities and food service helped to create a productive and efficient working environment. 8 4.00 0.00 Note. a Rated on a 4-point Likert scale with the response categories strongly disagree (1), disagree (2), agree (3) and strongly agree (4). Table 3.12 Evaluation of the standard setting process by the panellists Discussions During the standard setting process, a discussion took place that seems worth mentioning. In one of the groups, there was disagreement about the alignment of the profiles to the lowest language levels. While some of the panellists categorised a rather large number of profiles as pre-A1, other panellists classified only few profiles into this category. Most of the learners concerned had been able to successfully compose a very simple e-mail by using simple formulaic sentences such as My name is … or I like sports. In the story task, however, the learners seem to have had greater difficulties. The stories at these levels were more difficult to understand and contained considerably more German vocabulary. Many of the e-mails were thus rated as A1.1 while the stories of the same learners were considered as pre-A1. This difference between the two tasks made it difficult for the panellists to arrive at a final judgement. A similar observation was made at the highest levels, but vice versa. At these levels, it was often the story that was of higher quality. A joint discussion led to the conclusion that at the lowest levels more emphasis should be placed on the e-mail task, since the story might have been slightly too difficult for these learners, and that at the highest levels the story should receive more attention, since it may have given the strong learners more opportunity to display their competences than the e-mail task. Results After the panellists had entered their ratings in the online Excel sheet, it was first observed that the Excel sheet was only able to calculate two of the five cut scores, and that both were not statistically significant, probably because of the small number of panellists. An examination of the panellists’ ratings revealed that there was a considerable number of profiles that were, compared to their scores, rated unexpectedly high or low by several panellists. By excluding these 136 3 Methods profiles, cut scores could be calculated, but only one of them was statistically significant, namely the cut score between the levels A2.2 and above (p = .048). Furthermore, the impact data that was derived from these cut scores (see Figure 3.9) showed a distribution the experts did not have much confidence in. A comparison of the data with the rating scales which had been developed based on the CEFR language level descriptors confirmed this impression. According to the rating scales, a ‘typical’ learner at level A1.2 would have received a final score of 2.00 (see Table 3.1 and Table 3.2). The cut score between the levels A1.2 and A2.1, however, had a value of 2.02. Thus any profile with a score higher than 2.02 was classified as level A2.1 or higher. Level A1.2, as a consequence, almost completely disappeared because the two adjacent cut scores were very close to each other (1.76 and 2.02). Since level A1.2 represents the minimum requirement for EFL writing at primary school ( BKS , 2018; EDK , 2011), a much greater number of pupils was expected at this level. The impact data also did not comply with the results of other research studies carried out in Switzerland, which showed a much greater percentage of learners at level A1.2 (see Figure 1.1). For these reasons, the results of the first implementation of the standard setting were considered as not reliable enough to be used, and a detailed analysis was carried out in order to identify possible causes. Figure 3.9 Impact data of the first implementation of the standard setting Analysis and adaptations In order to identify the reasons for this unexpected result, the available data were examined in detail. First, the cut scores from the standard setting were 137 3.5 Standard setting compared to the expected values that had been derived from the rating scales. The analysis revealed that at the lower levels (pre-A1 and A1.1) the panellists’ classifications were generally more rigorous than expected, while they were more lenient than expected at the higher levels (A2.1 and above). This seems to have led to the cut scores for level A1.2 being very close to each other, and the number of pupils in this category being much smaller than expected. A second analysis was carried out that focused on the profiles that had been rated unexpectedly high or low by several panellists. The analysis revealed that salient features, in particular an orthography and a correctness of syntax and grammar that deviated from the average rating of the text, seem to have influenced the overall impression and thus the categorisation of the texts. All of the more closely analysed profiles that had been classified unexpectedly low by the panellists, for example, had been awarded scores for orthography that were considerably lower than the mean rating of the texts. Thus, the low level of orthography in these profiles may have had an influence on the overall judgement of the profiles. This observation seems to be in line with Vögelin, Jansen, Keller, and Möller (2018), who found that a low level of orthography in a text affected the perception of other text qualities, thus indicating a halo effect. A further observation was made by some panellists in their group discussion during the standard setting. They observed that the occurrence of cohesive devices such as because or but had led relatively quickly to a classification at level A2.1 and tempted the panellists not to study the text any further. This influence of salient text features may have been further intensified by the standard setting method, which required the panellists to holistically rate a rather large number of texts within a short time. Therefore, it was decided that the standard setting should be repeated, taking into consideration the insights from these analyses. First, a different standard setting method was selected, which did not require the panellists to rate many profiles within a short time, but allowed them to spend more time on studying fewer profiles. Secondly, rather than rendering a holistic judgement, the panellists were instructed to focus more specifically on the different text qualities and to discuss them thoroughly in order to avoid salient features dominating their judgements. Thirdly, the criteria for the selection of the profiles were adapted so that the difference between the two texts was smaller and the text qualities of the profiles more balanced. Fourthly, the panellists were instructed to pay more attention to the e-mail at the lowest levels, since the story might have been too great a challenge for these learners, and to consider that the e-mail task may have slightly restricted the good writers in displaying their abilities, and thus to more explicitly focus on the story for rendering their judgements. 138 3 Methods 3.5.3 Second implementation and final results Standard setting method For the second implementation of the standard setting, a different method was selected that largely followed the concept of the Booklet Classification Method (Loomis & Bourque, 2001) or Holistic Booklet Method (Hambleton et al., 2000). As in the Body of Work Method, the panellists do not focus on individual tasks but consider all pieces of a student’s work jointly (Hambleton et al., 2000), or a representative selection of items if there is a very large number of items (Loomis & Bourque, 2001). The panellists assign these booklets or profiles with the students’ work to the performance levels and / or try to determine the borderline candidates (Hambleton et al., 2000; Loomis & Bourque, 2001). According to Loomis and Bourque (2001), the profiles are sequenced in the order of their scores to facilitate the panellists’ task. The procedure described by Loomis and Bourque (2001) encompasses an initial classification (round one), feedback, discussions among the panellists and the opportunity to make changes to the classifications. The panellists try to reach agreement in these discussions and in round two they rate the profiles a second time (Loomis & Bourque, 2001). In contrast to this procedure, the standard setting applied in this study encompassed only one round with an extensive discussion that ended when the panellists had reached agreement on the least able candidate of their performance level. Selecting the profiles For the second standard setting, the criteria for the selection of the profiles were slightly adapted. The first two criteria (the number of raters and the rater difference) remained the same. Since the difference between the two tasks had led to difficulties at the lowest and highest levels, the third criterion, the task difference, was lowered from 0.5 to 0.25. However, in particular at the levels pre-A1 and A1.1 there were not enough texts that fulfilled this criterion. Therefore, it was nevertheless necessary at these levels to use profiles with a larger difference between the two texts. The panellists were instructed to consider this aspect and mainly focus on the e-mail if they observed large differences between the two texts (see discussion above). Besides these three criteria, a fourth criterion was added in order to reduce the influence of salient text features on the panellists’ judgements. Texts were excluded whose ratings for orthography, correctness of syntax and grammar or cohesion deviated markedly from their overall rating. Six profiles were selected for each cut score. Four of the profiles were located around the previously assumed cut scores or 139 3.5 Standard setting cut score ranges, and two slightly further away. Additional profiles were kept ready in case the panellists needed more profiles in order to come to a final judgement. If there were not enough profiles in a particular score range that fulfilled all criteria, profiles were added that fulfilled most criteria. Selecting and training the panellists Only six panellists could be found to participate in the second standard setting, in particular because it had to be conducted shortly before the end of the school year when both teachers and university staff were rather busy. The small number of panellists in this and also the first standard setting may have to be considered as a limitation of the standard setting process. However, the procedure of the second standard setting allowed work in groups so that two groups with three panellists could be formed, which appeared to be a suitable group size for the extensive discussions that were planned. As for the first standard setting, the panellists were selected based on their experience in the field of English language teaching. Four of them had been or were currently working as EFL teachers at primary school, and two at secondary school or at adult level. One was a professor for English language teaching, two were lecturers, one was a teacher trainer who ran professional development courses for primary school EFL teachers and two were highly experienced EFL teachers. All panellists were very familiar with the CEFR and had already used it for either practical, academic, scientific or assessment purposes. Again, the panellists were informed about the purpose and the programme of the standard setting prior to the event and provided with the CEFR performance level descriptors, the two writing tasks and two sample profiles. They were asked to study the documents and to highlight those elements in the CEFR performance level descriptors that would help to distinguish the different language levels from each other. At the standard setting, the jigsaw activity was left out since all panellists were very familiar with the CEFR and had worked with it many times before. After a brief introduction and some information about the purpose and procedure of the standard setting, the panellists were split in two groups. Each group was assigned to focus on a particular cut score. The panellists discussed in what way the two neighbouring language levels were distinct from each other, and took notes. In the survey to evaluate the standard setting process, the panellists indicated that the introduction had provided them with a clear understanding of the purpose of the meeting, and that the group discussions had aided their understanding of the CEFR language levels. 140 3 Methods Procedure After the discussion of the CEFR language levels, the panellists worked individ‐ ually and read the six profiles in order to identify the profile that would best represent the least able candidate of that particular language level. This was followed by an extensive discussion of the six profiles which lasted for about one hour. The profiles were discussed in detail, first by specifically considering the different CEFR level descriptors and then by directly comparing the profiles with each other. If necessary, further profiles were provided. The groups were asked to seek agreement on the least able candidate at their language level. If they did not agree, the average score was calculated to determine the cut score. This process was repeated until all cut scores had been defined. The procedural validity of the second standard setting was assessed with an evaluation form. As in the first standard setting, the panellists reported that they perceived the different standard setting components as adequately implemented. Results As can be seen in Table 3.13, the cut scores from the second standard setting were much closer to the expected cut scores than those from the first standard setting. Cut score Raw score of the cut score Standard setting 1 Standard setting 2 Expected Pre-A1 - A1.1 1.19 1.06 0.5-1.0 A1.1 - A1.2 1.76 1.56 1.5 A1.2 - A2.1 2.02 2.33 2.5 A2.1 - A2.2 2.88 3.30 3.5 A2.2 - above 3.38 3.70 3.7-3.8 Table 3.13 Cut scores from the two standard settings, compared to the expected cut scores After the panellists had been provided with the impact data of the second standard setting (see Figure 3.10), they were asked to complete an evaluation form and report their overall confidence in the defensibility and appropriateness of the final cut scores. On a scale from 1 to 4, they expressed an average confidence of M = 3.40 ( SD = 0.55) in the final recommended cut scores. 141 3.5 Standard setting Figure 3.10 Impact data of the second standard setting Additionally, they were asked to report on their confidence in the defensibility and appropriateness of the specific cut scores they had worked on. As can be seen in Table 3.14, the most difficult cut score seems to have been the one separating the levels A1.2 and A2.1. Cut score Level of confidence a n b M SD Pre-A1 - A1.1 2 4.00 0.00 A1.1 - A1.2 3 3.67 0.58 A1.2 - A2.1 5 2.80 0.45 A2.1 - A2.2 2 3.50 0.71 A2.2 - above 2 4.00 0.00 Note. a As assessed on a 4-point Likert scale with the response categories not confident at all (1), somewhat confident (2), confident (3) and very confident (4). b One panellist had to leave early and thus did not complete the evaluation form. The last cut score (level A1.2 - A2.1) was determined by the two groups jointly. Table 3.14 The panellists’ confidence in the defensibility and appropriateness of the final recommended cut scores 142 3 Methods The panellists discussed the profiles for this cut score thoroughly but at first did not seem to be able to come to a final conclusion. One of the reasons for the difficulties the panellists encountered may have been the selection of the profiles. Since there were only very few profiles around this expected cut score that fulfilled all criteria, several profiles had to be included which had a higher difference between the two texts than specified in the selection criteria. Since the second panellist group had already finished their task, they also studied the profiles for this cut score and joined the discussion. Additionally, further profiles were provided to the panellists. Eventually, the panellists were able to identify a score range (2.2-2.45) rather than a specific profile which they agreed upon to be borderline. The midpoint of this range (2.325) was thus determined to be the final cut score. 3.6 Data analysis 3.6.1 Descriptive text analyses Detailed descriptions of text quality differentiated by language level The detailed descriptions of young EFL learners’ texts (see chapter 4.1.3) were aimed at providing the reader with information about the different pragmatic, sociolinguistic and linguistic qualities of young EFL learners’ texts. They were intended to complement the quantitative findings presented in chapters 4.1.1 and 4.1.2 and give a more comprehensive picture of the learners’ EFL writing competence at the different language levels. For the analysis, six profiles (one per language level) were selected which had a final writing score in the middle of the respective language level (no borderline texts). Since the standard setting had revealed a certain influence of salient features on the overall impression of a text (see chapter 3.5), profiles were selected with a largely homogenous rating to allow, as far as possible, for a clear illustration of the text characteristics at the different language levels. However, because within-text heterogeneity was fairly common in the learners’ texts, two further profiles were selected for a detailed analysis of within-text heterogeneity (see chapter 4.1.4). The analyses of the different profiles were based on what is known from research about the different qualities of young EFL learners’ texts (see chapter 2.8) and on the rating scales developed for this project (see chapter 3.2.1). 143 3.6 Data analysis Analysis of the communicative effect in young EFL learners’ narrative texts Because descriptive findings about the communicative effect in young EFL learners’ texts are rare or do not exist (see chapter 2.8), a small qualitative analysis of this dimension was carried out in this study (see chapter 4.1.5 for the results). Its aim was to illustrate in what way young EFL learners create a communicative effect in narrative texts, and thus focused on the means the learners used to create such an effect. From each language level five texts were randomly selected whose scores for communicative effect / creativity represented the overall distribution of scores for communicative effect / creativity at that particular language level. Since only two learners had reached the language level above A2.2, this level was excluded from the analysis. This resulted in a total number of n = 25 narrative texts that were selected for the analysis. Data were analysed with MAXQDA Analytics Pro 2020 (Version 20.0.1). First, a pre-analysis with five texts was carried out in which all elements that were considered to create a communicative effect were marked. This pre-analysis was used to develop preliminary categories for coding the texts, which were again slightly adapted and extended during the main analysis. The texts were analysed by only one rater without checking for inter-rater reliability. Analysis of coherence in young EFL learners’ narrative texts Similarly, a small qualitative analysis of the coherence in the young EFL learners’ narrative texts was performed. Five texts from each language level were randomly selected whose scores for coherence represented the overall distribution of the scores for coherence at the corresponding language level. Since only two learners had reached the language level above A2.2, this level was excluded from the analysis. This resulted in a total number of n = 25 narrative texts that were selected for the analysis. Different approaches to analysing text coherence were reviewed (Butler & Zeng, 2014; Pon & Bagarić Medve, 2017). However, none of them appeared to be suitable for the intended analysis. The approach used by Butler and Zeng (2014), which was based on a model of story complexity originally presented by Stein and Policastro (1984), seemed to allow too little variability due to a strict sequence of categories. A story with an obstacle but without an ending, for example, was rated higher than a story with an ending but without an obstacle. This strict sequence did not seem to do justice to the variability observed in the children's texts. The approaches presented by Pon and Bagarić Medve (2017), on the other hand, did not seem suitable for providing EFL teachers with information that could be used for teaching and assessing EFL writing. Thus, an own simple procedure was developed that focused on two different aspects of coherence, namely the genre-specific 144 3 Methods structure of simple narrative texts, and different types of gaps and incoherence in the storyline (see chapter 4.1.6). The genre-specific structure of the texts was analysed using the coding functions in MAXQDA Analytics Pro 2020 (Version 20.0.1). As in the analysis of the communicative effect, coherence was analysed by only one rater without checking for inter-rater reliability. 3.6.2 Analysis of interview data with MAXQDA After the transcription, the data from the learner interviews were coded according to the topics of the research questions using MAXQDA Analytics Pro 2020 (Version 20.0.1), and analysed for information to further specify or exemplify the data from the teacher and learner questionnaires (see chapter 4.2.2 for the results). 3.6.3 Statistical analyses with SPSS Pairedand independent-samples t-tests, one-way and mixed between-within analyses of variance ( ANOVA ), Kruskal-Wallis and Wilcoxon Signed-Ranks tests, and regression analyses were realised with IBM SPSS Statistics (Version 25) according to different statistics manuals by Laerd Statistics (https: / / statistic s.laerd.com). 3.6.4 Multilevel analysis with MLwiN Since the data had a multilevel structure with pupils nested within classes, a multilevel analysis was conducted in order to examine the effects of different educational factors on the learners’ EFL writing competence. The analysis was realised with ML wiN software, version 3.05 (Charlton, Rasbash, Browne, Healy, & Cameron, 2020). As discussed in chapter 3.4.4, the large variability in the learners’ responses to questions about classroom practices did not allow for the analysis to be based on data from the learners. The analysis was therefore conducted using only the teachers’ responses to these questions (see chapter 4.3.2). 145 3.6 Data analysis 4 Results 4.1 Part I: The young EFL learners’ writing competence 4.1.1 The learners’ CEFR language level in EFL writing in grade six (RQ I.1) What CEFR language level do the learners reach in EFL writing at the end of primary school, and what percentage of learners achieves the minimum requirements as stated in the Swiss national standards and the cantonal curriculum? Figure 4.1 presents the distribution of the learners’ EFL writing competence by CEFR language levels (for information about the rating scales, the rating procedure and the setting of cut scores see chapters 3.2.1, 3.4.3 and 3.5). Note. The learners’ writing competence as measured at the time of data collection 4-6 months before the end of the school year (grade 6). N = 322 including 6.2 % learners with learning difficulties (e. g. dyslexia or individual learning objectives) and 2.2 % learners with close contact to the English language (e. g. L1 English or attendance of an English-speaking school). Figure 4.1 Distribution of the learners’ EFL writing competence by CEFR language levels As can be seen in Figure 4.1, the majority of learners reaches the CEFR language levels A1.2 (37 %) and A2.1 (46 %). About 10 % of the learners are at level A1.1 or below and about 7 % at level A2.2 or higher. Included in these figures are learners with learning difficulties such as dyslexia as well as learners with individual learning objectives (who do not have to reach the official curricular aims) and learners with close contact to the English language (e. g. L1 English or attendance of an English-speaking school). These pupils have not been excluded from the analysis since the study attempts to give a comprehensive picture of the EFL writing competence of all learners. If the pupils with learning difficulties are excluded from the analysis, the percentage of learners who did not reach level A1.2 decreases from 10.3 % to 7.6 %. The Swiss national standards for foreign language learning ( EDK , 2011) as well as the curriculum of the Canton of Aargau ( BKS , 2018) define level A1.2 as the minimum requirement for EFL writing at the end of primary school. Therefore, about 90 % of the learners reached or exceeded this level at the time of data collection (about 4-6 months before the end of the school year), while about 10 % did not yet achieve it. A comparison of these results with the findings by Bayer and Moser (2016) from a study in the same canton, as well as a discussion with regard to the national and cantonal requirements for EFL writing ( BKS , 2018; EDK , 2011), are presented in chapter 5.1.1. 4.1.2 The learners’ writing competence differentiated by their future educational track at lower secondary school (RQ I.2) What differences in terms of EFL writing competence can be observed when the learners are grouped according to their future educational track at lower secondary school? In order to provide the secondary school teachers with information about what writing competence might be expected of their learners when they enter lower secondary school ( ISCED 2 according to UNESCO , 2012), the following paragraph analyses the learners’ writing competence with regard to their future educational track at lower secondary school. In the Canton of Aargau, there are three educational tracks at lower sec‐ ondary school, namely the Realschule (low), Sekundarschule (medium) and Bezirksschule (high). Learners with individual learning objectives, who do not have to achieve the official curricular aims, either attend the regular Realschule, supported by a special needs teacher, or a so-called Kleinklasse with a reduced 148 4 Results class size for children with special needs. Three of the learners participating in the study were about to enter a Kleinklasse, 52 the Realschule, 108 the Sekundarschule and 148 the Bezirksschule. Eight learners did not yet know which educational track they would attend, two learners indicated that they would either attend a private school or move to another country, and one learner was absent when the pupils completed the questionnaire. A one-way between-subjects ANOVA was conducted to assess whether there is a statistically significant difference in EFL writing competence between the learners about to enter the different educational tracks at lower secondary school. The three different categories Kleinklasse / Realschule (n = 55, M = 1.86, SD = 0.54), Sekundarschule (n = 108, M = 2.28, SD = 0.56) and Bezirksschule (n = 148, M = 2.67, SD = 0.50) were used for the ANOVA analysis. There were three outliers in the category Kleinklasse / Realschule, as assessed by boxplot. They did not appear to be data entry or measurement errors and were thus considered as genuinely unusual data points. The one-way ANOVA was run twice, once with and once without the outliers. Since the results of the two analyses were similar with regard to test significance, post-hoc tests and effect size, the outliers were not removed. Data was not normally distributed in the category Kleinklasse / Realschule as assessed by Shapiro-Wilk test (p = .011) due to the three outliers, but normally distributed in the other two groups (p > .05). There was homogeneity of variances, as assessed by Levene's test of homogeneity of variances (p > .05). The analysis showed a statistically significant difference, F(2, 308) = 49.80, p < .001, with an effect size of η² = .24, which is a large effect size according to Cohen (1988). For post hoc comparisons, Hochberg’s GT 2 test was used because of the unequal number of learners in the three groups. It showed highly significant differences between the mean scores of all three categories (p < .001). These findings suggest that the different educational tracks at lower secondary school may indeed have to expect different average writing competences of the learners entering their schools. These statistically significant differences can also be observed in Figure 4.2, which gives a more detailed account of the learners’ writing competence differentiated by future educational track and language level. The majority of learners entering the Kleinklasse or Realschule (white bars) has an EFL writing competence at level A1.2, the majority of learners entering the Sekundarschule (dark grey bars) at level A1.2-A2.1 and the majority of learners entering the Bezirksschule (light grey bars) at level A2.1. However, the graph also shows that there is a large range of EFL writing competence within each track. Thus, all educational tracks may have to expect a large variety of EFL writing proficiency levels ranging from below A1.1 to A2.2 in 149 4.1 Part I: The young EFL learners’ writing competence the Kleinklasse / Realschule and from A1.1 to above A2.2 in the Sekundarand Bezirksschule. Figure 4.2 Distribution of the learners’ EFL writing competence differentiated by future educational track and language level A discussion of the implications of these results for teaching EFL writing at primary and secondary school, as well as their relevance with regard to the national and cantonal requirements for EFL writing are discussed in chapters 5.1.1 and 5.1.2. 4.1.3 Text samples and detailed descriptions of text quality differentiated by CEFR language level (RQ I.3) What are the characteristics and qualities of young EFL learners’ texts at different language levels? In order to illustrate the writing competence of young EFL learners at different language levels, sample profiles will be analysed and described in the following section. Each profile consists of the two texts written by the learners, a story and an e-mail. The descriptions analyse the different dimensions of text quality as discussed in chapter 2.8 and are related to the rating scales used for assessing the learners’ texts (see chapter 3.2.1). The two writing tasks can be found in chapter 3.2.2 and the detailed ratings of the six profiles in Appendix D. The children’s names and further data that could have been used to retrace their identity have been changed for reasons of data protection. 150 4 Results The profiles have been selected based on their ratings (see Appendix D) in order to represent typical performance at the different language levels. The profiles are fairly homogenous, both between the two texts and within the texts with regard to the different dimensions of text quality. This homogeneity appeared important in order to give the reader a clear illustration of the text characteristics and qualities typical for certain CEFR language levels. Homogeneity, however, does not seem to be an inherent feature of young EFL learners’ texts. On the contrary, their texts often appear to display considerable heterogeneity. Thus, chapter 4.1.3 first aims at providing a detailed description of typical text qualities at the different language levels, followed by an analysis and discussion of text heterogeneity in chapter 4.1.4. Profile 1: Approaching A1.1 The texts of Profile 1 were written by a learner with individual learning objectives in the school language and in English. Even though the language in these texts is limited, it can be observed that the learner is able to convey some very simple content. In particular, the e-mail covers most aspects the learners were required to write about and addresses the reader in a very simple way with a salutation and a question directed at one of the recipients (Hi and Jacob wi age? ). In the story, however, the writer does not fully succeed in developing a storyline, even if the parts written in German are considered. The storyline is incomplete and sometimes unclear. In terms of cohesion, the learner uses the additive conjunction and to link sentences. The English sentences are short and often incomplete. The German parts of the story, however, show that the learner is able to formulate sentences with subordinate clauses (the mom sieht dous er nicht do is), compound-complex sentences with multiple independent clauses and a subordinate clause (e.g. the dad is food and er sieht das der boy not hir is) and direct speech (the mom sagt geh sucht Dad) in the school language. Therefore, it seems likely that the brevity and incompleteness of the English sentences is due to limited language resources in the foreign language (see chapter 2.5), or the result of difficulties in accessing them (see chapter 2.6). It can also be observed that the learner uses knowledge of the school language to formulate sentences in English (e.g. i … go football Play - Ich gehe Fussball spielen). The vocabulary is limited and large parts of the story written in the school language. Nevertheless, it can be observed that the learner knows some simple English words such as mom, dad, boy, food, book, look, not, my, I, go, play or football. Furthermore, the learner appears to have successfully copied the expressions school subjects and age from the task sheet and applied them in the right context. Several strategies to overcome or deal with the challenge of limited L2 vocabulary can be observed, namely the use of German words (i go … in Kino [=cinema]), the use of an English substitute (the dad is food) and omission (i 13 age). 151 4.1 Part I: The young EFL learners’ writing competence The orthography is limited but in most cases good enough to be comprehensible. As also observed by Vogt and Bader (2017), a considerable number of one-syllable words (e.g. dad, food, boy), frequently used words (e.g. the, is, and) and words with the same phoneme-grapheme correspondence in the two languages (e.g. is, in) are written correctly. Capitalisation, however, does not yet seem to be stable. Regarding punctuation, it can be observed that besides one full stop at the end of the story, no punctuation has been used. Note. Grey font = words in the school language. Blue font = expressions the learner may have copied from the task sheet. Profile 1: Learner approaching level A1.1 In summary, the author of Profile 1 shows in the e-mail that he is able to ‘communicate very basic information about personal details in a simple way’ (see Appendix A, Table A2, propositional precision, level pre-A1) and can establish very basic social contact. In terms of language, he uses ‘isolated words and basic expressions in order to give simple information about him / herself ’ (see Appendix A, Table A4, general linguistic range, level pre-A1). Limited language resources or difficulties in accessing them appear to constrain the learner’s ability to express his ideas. Profile 2: A1.1 The author of Profile 2 seems to have more resources available than the learner before, even though the language in the texts is still limited. In both texts, he displays the ability to communicate basic information. The e-mail covers most aspects that were asked for, even though without much detail. The text addresses the reader in a very simple way and contains three typical elements of an e-mail, namely a salutation (Hey), questions directed at the recipients (e. g. Going tu 152 4 Results school? ) and a complimentary close (Bye). The story is comprehensible and with the direct speech “Mom common its cool” the learner succeeds in achieving a very small communicative effect. The scene, people and actions, however, are described without much detail and the writer does not seem to fully succeed in developing a storyline. Note. Grey font = words in the school language. Blue font = expressions the learner may have copied from the task sheet. Profile 2: Learner at level A1.1 The learner uses few very basic cohesive devices (and, and then) to link words and sentences, yet the texts mainly consist of isolated phrases and sentences. The sentences are short and, in particular in the e-mail, often incomplete with some reduction and omission of elements (e. g. the mater coing, Like Realien and Germany or Going tu school? ). Similarly to Profile 1, the influence of the school language on L2 syntax is recognisable (e. g. The Kid coms not beak. - Das Kind kommt nicht zurück.). In contrast to Profile 1, however, most of the text is written in English (except for the term Realien, which is a school subject that encompasses history, geography, biology, technology, humanity and economy, see BKS , 2016, p. 215). Nevertheless, the vocabulary is limited. Several passages show how the learner is trying to deal with the limited vocabulary resources. Similarly to the author of Profile 1, he uses English substitutes (e. g. The fater and the mater food or hocky is my love sport.), omission (e. g. of personal pronouns such as I or you), and copies some phrases from the task sheet (e. g. What you like in your free time? and What school subjects you like why? ). In terms of 153 4.1 Part I: The young EFL learners’ writing competence orthography, there are some words that might be difficult to understand if the context is not taken into account (e. g. ried = read, beak = back). In general, however, the text appears to be comprehensible even if the learner seems to have only limited control and the text contains various inconsistencies. Some influence of the school language on orthography can be observed (e. g. fater = Vater in German; or the capitalisation of nouns, e. g. Book or Kid). Similarly to Profile 1, several one-syllable words (e. g. food, say, bring) are spelt correctly, as well as some frequently used words (and, not) and words with the same phoneme-grapheme relationship as in German (is, in). Punctuation is mostly correct and the learner even uses quotation marks for direct speech, even though in German notation. In summary, the author of Profile 2 displays the ability to convey basic information about personal details … in a simple way (see Appendix A, Table A2, propositional precision, level A1) and to establish basic social contact. He ‘has a very basic range of simple expressions about personal details’ and ‘can use some basic structures in one-clause sentences with some omission or reduction of elements’ (see Appendix A, Table A4, general linguistic range, level A1). Profile 3: A1.2 The author of Profile 3 exemplifies a learner with a writing competence at level A1.2, which is the minimum requirement of EFL writing at the end of primary school ( BKS , 2018; EDK , 2011). 37 % of the learners participating in this study reached this level (see chapter 4.1.1). As can be seen in Profile 3, the learner clearly succeeds in conveying relevant content and information. It is evident that the learner is trying to tell a story, even though there seem to be some gaps (e. g. what mum does after Jacob left to find his father), and it is sometimes not immediately clear who is speaking because of missing reporting clauses for direct speech. The writer describes the scene, people and actions with a few details (e. g. what people say) and uses different elements to catch the reader’s attention: She starts the story directly with a question (“How is Jacob? ”), creates some tension (One moment it was still, …) and includes direct speech several times (e. g. “Jacob, common we eating now! ”). Similarly, in the e-mail the learner responds to all aspects of the writing prompt, provides basic information about herself and conveys a few details about one or two aspects (why she likes the school subjects Sports and English and what her hobbies are). She addresses the reader in a simple but effective way: there is a salutation (Hello), she asks questions and directly addresses the two recipients (Jacob, wath is …? Sophie, wath make you …? ), and she finishes the e-mail with a complimentary close (Loves and kisses from Switzerland) and her name. 154 4 Results Note. Blue font = expressions the learner may have copied from the task sheet. Profile 3: Learner at level A1.2 In terms of coherence, there is a simple sequence of topics in the text. Regarding cohesion, the writer mainly uses the additive conjunction and and the temporal conjunction then to link words and sentences. The causal conjunction because, however, does not yet seem to have been mastered (e. g. I like English why I will go and study in America.). In comparison to the authors of Profiles 1 and 2, the writer already uses some personal pronouns such as he, hes (= his), you and we. The possessive pronoun my is not yet used correctly (me name is Selina or in me free time), which may either be a confusion of the pronouns me and my or also a misspelling. Additionally, the learner uses the two temporal expressions 10 min. laiters and at the end of the story. She mainly uses simple sentences but in contrast to Profiles 1 and 2, there is much less omission and reduction of elements. Nevertheless, the control of syntax and word grammar is still limited (e. g. Wath make you in your free time? or the use of tenses), and there are several inconsistencies such as Common, we are eating! (lines 1 and 2) and Jacob, comon 155 4.1 Part I: The young EFL learners’ writing competence we eating now! (line 3). Also in this profile, some influence of the school language on L2 syntax can be observed (What make you in your free time? - Was machst du in deiner Freizeit? ). The learner appears to have a large enough repertoire of basic vocabulary to be able to write both texts in English. However, there are still some words that seem to indicate some vocabulary constraints (e. g. How is Jacob? instead of Where is Jacob? , nofing instead of nobody or will instead of want to). While nofing appears to be an L2 substitute, both How and will are false friends: The German word for Where? is Wo? , and the German word for want to is will. Regarding orthographic control, there are still limitations (e. g. vather, wath) but it can be observed, in particular in the e-mail, that the learner succeeds in writing many short and very common words with reasonable phonetic accuracy. In addition, some influence of the school language on spelling can be observed (vather = Vater in German). Basic punctuation is mostly correct, and this learner also uses quotation marks for direct speech (in German notation). In summary, the author of Profile 3 clearly succeeds in communicating ‘basic information about personal details … in a simple way’ (see Appendix A, Table A2, propositional precision, level A1) and in establishing ‘basic social contact by using the simplest everyday polite forms’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A1). The texts also show that the learner ‘has a basic vocabulary repertoire of words and phrases related to particular concrete situations’ (see Appendix A, Table A4, vocabulary range, level A1). Profile 4: A2.1 The current curriculum of the Canton of Aargau specifies the minimum requirements the learners have to reach, but also explicitly states that the pupils are supposed to work on the next higher levels when they have accomplished the basic level ( BKS , 2018, p. 20). Profile 4 illustrates the writing competence of a learner at such an enhanced level, namely at level A2.1. As presented in chapter 4.1.1, about 46 % of the learners participating in this study reached this level. As can be seen in Profile 4, this learner succeeds in telling a simple story and conveying basic information about herself. The story is coherent and contains several elements of a simple narrative structure: setting the scene (Last Week Family Müller wond eat lunch butt Max the son were not by the Table.), development and complication (first Max and then Dad reading the book while the others wait for having lunch) and a story ending (Max and Dad reading the book together). What is missing is a resolution of the problem that Mum is still waiting for them in the kitchen. The learner describes the scene, people and actions with some details (e. g. Max the son were not by the Table. Becors hes reading a fascinating book.). The text has a small communicative effect, 156 4 Results containing some elements that catch the reader’s attention (e. g. the reader having more information than Dad when he asks “Wer is Max? ” or the writers’ consternation Now Max is sitting on the table but his Dad is not thear! ). Note. Grey font = words in the school language. Blue font = expressions the learner may have copied from the task sheet. Profile 4: Learner at level A2.1 The e-mail responds to most aspects of the writing prompt: while the author does not give any reason why she likes her favourite school subjects, she adds some further elements to the e-mail, namely the pets she has got and what her favourite animal is. Her hobbies, trampoline jumping and swimming, are described in some detail (I have an one Trampolin his sice is three meters i like it very much and I’m going all second Friday swimming with my Dad in Hallenbad, thear it is nice and worm). The only aspect that is considerably lower than expected at this level is the genre-specific elements of an e-mail. 157 4.1 Part I: The young EFL learners’ writing competence While there is a salutation (Hello) and some questions introduced by the phrase Now I have some questions to you to, other key aspects such as a concluding sentence, a complimentary close or the sender’s name are missing. In terms of cohesion, it can be observed that the author of Profile 4 uses a wider variety of cohesive devices and reference words than the authors of Profiles 1 to 3. Besides additive (and, or, also), temporal (last week, now), causal (because) and adversative conjunctions (but), she also uses a variety of personal pronouns correctly (he, his, him, my, you). She does not yet, however, use the pronouns its (she uses his to refer to the trampoline), your and mine (Wats our favourit anymal? my is the Tiger). In general, the two texts are linguistically mostly well linked. In terms of syntax and word grammar, the writer uses a mixture of simple and more complex sentences and grammatical structures. The learner, for example, uses compound-complex sentences (e. g. He told him that the lunch is going cold but then he lied down too his Dad and read in the book with him! ) or formulates a question with do you (Do you have some anymals? ). She often uses the tenses present simple and present continuous (Max is going inside and sit down), tries out some past tense forms (e. g. He told him or he lied down) and once uses a 3 rd -person -s (his Dad reads in his book! ). These forms, however, do not yet seem to be stable. The learner uses a few simple structures correctly but still systematically makes basic errors. Some influence of the school language can be observed in terms of word order in sentences: I’m going all second Friday swimming (= Ich gehe jeden zweiten Freitag schwimmen). Regarding vocabulary, the learner mainly uses basic vocabulary but also a few specific vocabulary items that allow a slight elaboration of the text (e. g. inside, autside, sice (= size) or lesson). In contrast to Profile 3, the question word where is used correctly, although not spelt correctly (Wer is Max? ). Several strategies to overcome the challenge of limited vocabulary can be observed: using the school language (Hallenbad = swimming pool), borrowing words from the school language and adapting them so that they look like English words (by the Table - bei dem Tisch, by and bei having a similar pronunciation; or the Mum meand from die Mutter meint) and L2 substitutes (on the table instead of at the table; or my is the tiger instead of mine is the tiger). In terms of orthography, it can be observed that short and very common words are written with reasonable phonetic accuracy. The learner still makes basic errors (e. g. becors = because or jears = years), but she also displays the ability to write some longer words (e. g. reading, sitting, jumping, swimming) and some words with difficult phoneme-grapheme correspondence (e. g. down, Friday, draw or questions) correctly. The capitalisation still appears to be challenging (e. g. Week, Family, Name, i, Trampolin). The contraction I’m with an apostrophe, however, is used correctly. Basic punctuation is in large part 158 4 Results used correctly, except for a redundant full stop in line 1 of the story and three missing full stops in the e-mail (in lines 2 and 7). As in the previous profiles, the learner uses the German notation of quotation marks. In summary, Profile 4 shows that this learner is able to ‘communicate what he / she wants to say in a simple and direct exchange of limited information on familiar and routine matters’ (see Appendix A, Table A2, propositional precision, level A2 / A2+). She ‘can establish basic social contact’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A1) and ‘has a sufficient vocabulary for the expression of basic communicative needs’ (see Appendix A, Table A4, vocabulary range, level A2). She ‘uses some simple structures correctly, but still systematically makes basic mistakes’ (see Appendix A, Table A4, grammatical accuracy, level A2 / A2+) and ‘can write with reasonable phonetic accuracy … short words that are in his / her oral vocabulary’ (see Appendix A, Table A4, orthographic control, level A2 / A2+). Profile 5: A2.2 Profile 5 represents a learner at the CEFR language level A2.2. Learners who reach this level in EFL writing by the end of primary school are not as numerous as the learners at the two previous levels. About 6 % of the learners participating in the study reached level A2.2 (see chapter 4.1.1). As can be seen in the texts of Profile 5, the learner succeeds in telling a simple and coherent story and is able to convey detailed information about herself in an e-mail. The story has a simple but clear communicative effect and contains several elements that make the story interesting. The fact that dad is not surprised to find his daughter reading as well as him saying with a deep voice ‘enough reading its dinner time’ and later ‘Okay i am coming sweeti! ’ are all elements that catch the reader’s attention. Moreover, the learner creates some tension by first having dad hide the fact that he is reading a book about Barbies, and eventually finishes the story with dad not even being willing to stop reading at the dinner table. The scene, people and actions are described in detail (e. g. Dad walking through the living room, opening the door or speaking with a deep voice). The story has a simple and coherent storyline and contains most elements of a simple narrative structure. There is a brief setting of the scene (although only the time is introduced and the reader is not immediately informed about the meal or about who Hans is). The story then develops, culminating in a complication when dad tries to hide what he is doing, followed by a resolution (dad admitting that the book is interesting), and finally ending when everybody is having dinner. 159 4.1 Part I: The young EFL learners’ writing competence Note. Blue font = expressions the learner may have copied from the task sheet. Profile 5: Learner at level A2.2 The e-mail similarly responds to all aspects of the writing prompt, adds two further elements (where Emilia lives and what animals she has got) and describes two hobbies in detail (playing with friends and listening to music). The text has the typical form of an e-mail and addresses the reader in a simple but effective way. There is a salutation (Hello Sophie and Jacob), the writer asks questions and directly addresses the two recipients (Whats yours favorite subjects Sophie and Jacob? ), makes a request (Would you guys write me back? ) and finishes the e-mail with a complimentary close (Love), her name and a smiley. In the story, an additional aspect of social contact, a polite request (Could you pleas get her here? ) can be found. In terms of coherence, the e-mail is 160 4 Results structured as a simple and clear sequence of topics. Regarding cohesion, the two texts show a large variety and amount of cohesive devices and reference words, in particular the e-mail. The story uses a smaller range and is lacking some reporting clauses for direct speech, but generally the texts are linguistically well linked. The learner uses different types of conjunctions, namely additive (and, or), temporal (when), causal (because) and adversative conjunctions (but). Furthermore, she uses demonstratives (this, that, those, here, there), pronouns (I, me, my, you, your, yours, he, she, her, it, its, we, they, them) and expressions such as you two, one of them and the other one. In terms of syntax and word grammar, the author of Profile 5 uses more complex and varied sentences and grammatical structures than the previous profiles. Throughout the story, the learner uses the past tense (both in simple and continuous form) and clearly distinguishes it from the present tenses used in direct speech. This seems to be in line with the findings by Hasselgreen and Sundet (2017), who found that young learners at CEFR level A2 used a wider range of past tense forms than learners at level A1 did (p.207). The author of Profile 5 also shows some awareness of how to formulate polite requests (Could you pleas…? Would you …? ), formulates questions with Do you …? (Do you have any animals? ) and uses the passive voice (one of them is called Karl). Besides simple sentences, there are also compound sentences (e.g. I have two bunnys, one of them is called Karl and the other one is Katharina), complex sentences with a subordinate clause (e.g. I like those subjects because they are very fun.) and compound-complex sentences (e.g. I like playing or listen to music because when i play with friend i don’t feal lonly and i can talk with them about everyting.). The writer uses some simple structures correctly but still makes basic errors (e.g. Lisa where you doing here? or Whats yours favorite subjekts Sophie and Jacob? ). An influence of the school language on L2 syntax as in previous profiles, however, cannot be observed. In terms of vocabulary, the learner uses a lot of simple vocabulary but also some specific and varied terms that allow some elaboration of the text (e.g. suprised, deep voice, enough reading, sweeti, laugh very hard or feal lonely (= feel lonely)). Also, the selection of vocabulary does not show much influence of the school language and the texts do not seem to allow any conclusions as to how the author dealt with a limited L2 vocabulary. Regarding orthography, short and very common words are written with reasonable phonetic accuracy but the learner still makes basic errors (e.g. wose = was, dinig room = dining room or intrestin = interesting). On the other hand, there are already many difficult words that are written correctly (e.g. o’clock, where, could, voice, enough, saw, nothing, laugh, draw or would). Basic punctuation is in large part used correctly, even though some influence of the school language can be observed (e.g. use of quotation marks). In summary, the author of Profile 5 shows the ability to ‘tell a story or describe something in a simple list of points’ (see Appendix A, Table A2, 161 4.1 Part I: The young EFL learners’ writing competence thematic development, level A2+) and to ‘socialise simply but effectively using the simplest common expressions and following basic routines’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A2+). She ‘can use the most frequently occurring connectors’ (see Appendix A, Table A2, coherence and cohesion, level A2+) and ‘has a repertoire of basic language, which enables him / her to deal with everyday situations with predictable content, though he / she will generally have to compromise the message and search for words’ (see Appendix A, Table A4, general linguistic range, level A2+). Profile 6: Above A2.2 Profile 6 is an example of a learner who exceeds level A2.2 in EFL writing and clearly surpasses the curricular expectations. As presented in chapter 4.1.1, only 0.6 % or two of the learners participating in the study reached this level. As can be observed in the texts of Profile 6, the learner succeeds in telling an interesting and detailed story and in communicating effectively with the recip‐ ients of her e-mail. The story has a simple communicative effect and contains several elements that make the story interesting (e. g. Tom being disappointed that he has to stop reading or being angry that he has to interrupt his meal). The learner also creates some tension when describing Tom’s astonishment when he finds his dad reading the book (At the moment I got in my room, I saw my dad at the floor and reading my book. That wasn’t right! I had to stop reading but he was alaud to read? No! ), and when dad ignores him and refuses to go to the kitchen. The scene, people and actions are described in detail and the storyline is coherent and contains all elements of a simple narrative structure: There is a clear setting of the scene that introduces the time, people, place and current situation. There is a clear development that culminates in a complication when dad ignores Tom and continues reading instead. A further complication evolves when the mother, who had been waiting for a long time, appears in the room. Unexpectedly, however, she starts reading herself after the others have returned to the kitchen, and the situation is finally resolved when Tom realises the funny side of the situation and the captivating effect of the book. In the e-mail, the author responds to all required aspects and adds several further elements (her dog, her favourite colour, her favourite food and what she wants to do when she is older). The e-mail conveys detailed information and addresses the reader in a simple but effective and appropriate way. There is a salutation which directly addresses the two recipients (Hi Sophie, Hello Jacob.), the author asks questions and finishes the e-mail with a concluding sentence (Well, I hope to write soon with you.), a complimentary close (Goodbye and have a nice day.) and her name. 162 4 Results Note. Blue font = expressions the learner may have copied from the task sheet. Profile 6: Learner above level A2.2 163 4.1 Part I: The young EFL learners’ writing competence The e-mail shows a slight elaboration of the text structure by naturally se‐ quencing different topics. The learner, for example, writes about her favourite school subject English and how she learnt the language and then moves on to the topic sports using the sentence I like sport also but not that much like english. In terms of cohesion, the texts are well linked. The learner uses a wide variety and large amount of cohesive devices and reference words, namely additive (and, also, or), temporal (while, therewhile, when, once), causal (so, cause, because) and adversative conjunctions (but). There are also demonstratives (that, there), comparatives (older, not that much, like) and pronouns (I, my, me, you, it, she, her, he, his, him, we, our, us, everyone), as well as expressions such as last week, suddenly, a long time, at the moment I got in my room, you two or the book that I were reading. Compared to the previous profiles, the learner uses more complex and varied sentences and grammatical structures. The learner uses both past simple and past continuous forms and appears to be well able to distinguish between them (e. g. My mom and me were siting at the table and suddenly I realised that my dad wasn’t there.). In one case, she uses the past perfect (I was a bit angry cause I just had started eating.). She also uses the passive voice (e. g. but he was alaud to read? or everyone was fascinated by it.), formulates negative sentences (e. g. That wasn’t right! , she didn’t came or My parents aren’t from England.) and questions with Do you …? (What do you think about the Space? ). While the e-mail mainly consists of simple sentences (e. g. I like the subjects English and Sport.) and a few compound (e. g. Once I had a dog, but he died.) and complex sentences (English is my favorite because I love to speak english.), the story contains to a large extent complex (e. g. It was so intresting that I forget the time.) and compound-complex sentences (e. g. Hi, I’m Tom and I wanna tell you a funny story that happened last week.) but only a few simple (e. g. My mom and dad were eating lunch.) and compound sentences (e. g. He was also intrested so he started reading.). Even though the learner still makes certain errors (e. g. My dad saw, that i didn’t was at my seat), she uses the sentences and grammatical structures ‘reasonably accurately’ (see Appendix A, Table A4, grammatical accuracy, level B1). In comparison to the previous profiles, the author of Profile 6 uses a wider range of vocabulary, which allows a clear elaboration of the text. Besides a broad range of simple vocabulary, there are also more advanced vocabulary items such as disappointed, suddenly, take a look, realise, being allowed to, amazing or fantastic. There are a few expressions that show some influence of the learner’s L1, such as lunch was finished (instead of lunch was ready), the use of stand up and get up as synonyms (in the German language, there is only one word for both concepts), by the table (= bei dem Tisch) or saw her on the floor by reading my book (from German: sah sie auf dem Boden beim Lesen meines Buches). In terms 164 4 Results of orthography, the learner shows a high level of orthographic control. There are only few words that are not written correctly (e. g. intresting, i, tabel, alaud to or self instead of shelf). Basic punctuation and some further elements such as exclamation marks and some commas are used correctly. Some influence of the school language can be observed in the use of commas for dependent clauses where the learner applies the German rules (e. g. My dad saw, that i didn’t was at my seat). In summary, the author of Profile 6 shows that she can ‘convey simple, straight‐ forward information of immediate relevance, getting across which point he/ she feels is most important’ (see Appendix A, Table A2, propositional precision, level B1) and ‘socialise simply but effectively using the simplest common expressions and follow basic routines’ (see Appendix A, Table A3, sociolinguistic appropriateness, level A2+). Furthermore, she displays the ability to ‘form longer sentences and link them together using a limited number of cohesive devices, e.g. in a story’ (see Appendix A, Table A2, coherence and cohesion, level B1), ‘has a good command of a range of vocabulary related to familiar topics and everyday situations’ (see Appendix A, Table A4, vocabulary range, level B1) and ‘can produce continuous writing, which is generally intelligible throughout’ (see Appendix A, Table A4, orthographic control, level B1). The relevance of knowing about these characteristics and qualities of young EFL learners’ texts for teaching and assessing EFL writing is discussed in chapter 5.1.3. 4.1.4 Heterogeneity in young EFL learners’ texts After this description of six sample profiles from different language levels, chapter 4.1.4 turns to another key characteristic of young EFL learners’ texts, namely their heterogeneity. The chapter first examines the heterogeneity within the learners’ texts, followed by an analysis of the heterogeneity as between the two types of text (e-mail and story). Heterogeneity within the texts Within-text heterogeneity is defined as the variability within a text with regard to its different text qualities, and was measured by the standard deviation of the different sub-ratings of a text from its mean rating. A low standard deviation thus indicates that a text is homogeneous with regard to the different dimensions of text quality. A high standard deviation, on the other hand, indicates a high variability between the different dimensions of text quality. 165 4.1 Part I: The young EFL learners’ writing competence As can be seen in Figure 4.3 and Figure 4.4, the within-text heterogeneity of the stories ranged between 0.30 and 1.36 (M = 0.70), and that of the e-mails between 0.34 and 1.17 (M = 0.71). Note. M = 0.70, SD = 0.20, n = 322 Figure 4.3 Within-text heterogeneity story Note. M = 0.71, SD = 0.17, n = 322 Figure 4.4 Within-text heterogeneity e-mail The six profiles presented in chapter 4.1.3 were fairly homogeneous in their ratings of the different dimensions of text quality so that the different language levels could be clearly illustrated. Their measures of within-text heterogeneity were below average ( SD = 0.35-0.70), except for the e-mail in Profile 1, which had a standard deviation slightly above average ( SD = 0.79). Figure 4.5 illustrates the within-text heterogeneity of Profile 3 (see chapter 4.1.3). The ratings of the different text qualities ranged 166 4 Results between 1.5 and 2.5 for the story (with a slightly higher rating for punctuation), and between 2.0 and 3.0 for the e-mail, with a standard deviation of 0.53 and 0.48 for the two tasks. As shown in Figure 4.3 and Figure 4.4, however, this homogeneity does not adequately represent the within-text heterogeneity of the whole sample. Thus, in order to provide a more comprehensive picture of the learners’ writing competence, two further profiles will be presented, both of which have a within-text heterogeneity which is above average. Note. Story: M = 2.06, SD = 0.53. E-mail: M = 2.30, SD = 0.48. Figure 4.5 Example of a profile with small within-text heterogeneity (see Profile 3, chapter 4.1.3) 167 4.1 Part I: The young EFL learners’ writing competence The first profile (Profile 7) represents a writing performance at level A1.2 (Rasch-adjusted final score = 2.22), and has a within-text heterogeneity of SD = 1.01 in the story and SD = 0.99 in the e-mail. This comparably high within-text heterogeneity may not be visible at first sight. The ratings, however, reveal that the complexity of syntax and grammar and the orthography in the story are considerably higher than would be expected at this level. In terms of syntax and grammar, it can be observed that the learner did not use only simple sentences (e. g. Lorik went to Juniors room.), but also compound sentences (e. g. Lorik and Annalena wanted to eat but Junior was not eat desk, he was in hes room.) and complex sentences (e. g. The father sad he nid to went to the eat desk.). He also used a variety of regular and irregular forms of the simple past (e. g. wanted, was, went, sad [= said], found or readed), negation in the past tense (didn’t found), the past continuous (Junior was reading a book.) and expressions such as wanted to or nid to (= need to). Note. Blue font = expressions the learner may have copied from the task sheet. 168 4 Results Note. M = 2.56, SD = 1.01 Note. M = 2.10, SD = 0.99 Profile 7: Learner at level A1.2, texts and rating profiles 169 4.1 Part I: The young EFL learners’ writing competence There are certain grammatical errors (e. g. mixing of tenses; he nid to went; readed; or he didn’t found the father), but as Lasagabaster and Doiz (2003) observed, such errors often only occur at higher levels, since the corresponding structures are used rarely or not at all at lower levels. In terms of orthography, there are only a few words that are not written correctly, mainly the words hes (= his), sad (= said) and nid (= need), as well as some words that need an apostrophe such as I’ts, fathers or Juniors. In the e-mail, the learner covered all required aspects and additionally mentioned his favourite food, thus receiving the highest rating for coverage. However, he only conveyed very basic informa‐ tion without much detail, both in the e-mail and in the story, and thus received only a low rating for this dimension. Similarly, coherence and punctuation were rated low in the e-mail, since the text is slightly repetitive (football, kickboxing) and basic punctuation is missing or incorrect several times. The second profile (Profile 8) has a within-text heterogeneity of SD = 0.87 in the story and SD = 0.84 in the e-mail, and shows an overall writing performance at level A2.1 (Rasch-adjusted final score = 2.36). This overall rating may appear surprising at first sight, in particular since the vocabulary range appears to be rather small, with many expressions in the German language (e. g. aber = but; keine = no; or lesen = read), and because of the limited orthography (e. g. Ivening = evening; whis = with; or geve hem = give him), which may sometimes affect comprehensibility. Also, the correctness of syntax and grammar in the e-mail is limited, with certain inconsistencies (e. g. it maks fun and it make fun), some reduction of elements (e. g. the sun shyning) and a clear influence of the German language (e. g. Maths like I = Mathematik liebe ich; or when the sun shyning go I = wenn die Sonne scheint, gehe ich). 170 4 Results Note. Grey font = words in the school language. Blue font = expressions the learner may have copied from the task sheet. 171 4.1 Part I: The young EFL learners’ writing competence Note. M = 2.28, SD = 0.87 Note. M = 2.60, SD = 0.84 Profile 8: Learner at level A2.1, texts and rating profiles 172 4 Results However, compared to Profile 7, the two texts are much more detailed. In the story, for example, the learner describes the time of the day; she informs the reader that the book was from the boy’s friends because his dad did not give him any books; or she mentions that the book was a comic. Similarly, the e-mail contains various details, for example, on what days the learner pursues her hobbies and with whom; or that she knows her friends Sina and Kim from school. In addition, punctuation is much more elaborate compared to Profile 7, with colons and double quotation marks for direct speech (which is the German notation), commas for series of words (e. g. I love Maths, TW , AW and Germany.) or to separate clauses (e. g. and lest in a book, that becoms he from hes frends, becouse hes dad geve hem keine books.) and introductory expressions (e. g. Com on, we mast gowing to the brakefast.). In the e-mail, further text qualities such as the use of genre-specific elements or coherence and cohesion were rated high. The e-mail contains several genre-specific elements such as a salutation that directly addresses the two recipients (Hello Sophie and Jacob), questions directed at the reader (e. g. What make you Jacob in the Sommer? ), a concluding sentence (Good bay), a complimentary close (Love) and the learner’s name (Mira). In terms of coherence, there is a simple and clear sequence of topics in the text and a clear transition from recounting the learner’s own summer activities (And in the Sommer when the sun shyning go i whis my Friends of the Trampolin and picnicen.) to asking Jacob what he does in summer (What make you Jacob in the Sommer? ). In terms of cohesion, the text is in the main linguistically well linked, with some additive (and, alsow), temporal (when) and causal conjunctions (becouse), as well as some pronouns (I, my, you, it) and the demonstrative this. In contrast to Profiles 1-6 (see chapter 4.1.3), Profiles 7 and 8 illustrate young EFL learners’ texts with a high within-text heterogeneity. As can be seen in these two examples, within-text heterogeneity may take different forms. While Profile 7 displayed a high level of accuracy and syntactical and grammatical complexity, Profile 8 displayed more communicative qualities, in that the content and plot were described and narrated in detail and that genre-specific elements were used well. On the other hand, Profile 7 showed a comparably low level of detail, coherence and punctuation, while Profile 8 was more limited in terms of vocabulary, orthography and syntactical and grammatical correctness. These observations appear to have important implications for classroom practice and research, which will be discussed in chapter 5.1.5. 173 4.1 Part I: The young EFL learners’ writing competence Heterogeneity between the text types Besides within-text heterogeneity, the texts written by young EFL learners also seem to display considerable heterogeneity as between the two types of text (e-mail and story). In order to investigate this heterogeneity, a paired samples t-test was conducted to determine whether there was a statistically significant difference between the final scores of the e-mail and the story. Three outliers were detected that were more than 1.5 box-lengths from the edge of the box in a boxplot. Two outliers were excluded (one text was written in German, and one learner had stopped writing after one sentence), while the third was not removed, since its between-text difference was not extreme and both texts were complete. The assumption of normality was not violated, as assessed by Shapiro-Wilk’s test (p = .608). The paired samples t-test showed no significant difference between the final scores of the e-mail (M = 2.38, SD = 0.60) and the story (M = 2.35, SD = 0.69), with a mean difference of 0.03, 95 % CI [-0.02, 0.07], t(319) = 1.17, p = .242, d = 0.07. During the standard setting process, however, between-text differences had been observed in particular at the lowest levels, where the learners appeared to cope better with the e-mail, and at the highest levels, where the learners seemed to perform better at writing the story. Thus, a mixed between-within ANOVA was conducted to assess whether there was a significant interaction between tasks and language level on the learners’ EFL writing competence. There were five outliers as assessed by boxplot and by examination of studentised residuals for values greater than ±3. They did not appear to be data entry or measurement errors and were thus regarded as genuinely unusual data points. The analysis was run twice, once with and once without the outliers. Since the results of the two analyses were similar with regard to the significance of interaction, the outliers were not removed. Because the data did not meet the requirement of homogeneity of covariance (Box's test of equality of covariance matrices, p < .001), Pillai’s Trace instead of Wilks’ Lambda was used to evaluate the significance of interaction (Tabachnick & Fidell, 2007). The analysis showed that there was a significant interaction between task and language level on the learners’ EFL writing competence, F(3, 318) = 7.36, p < .001, partial η 2 = .065, which is a medium effect size according to Cohen (1988). In order to analyse this interaction in more detail, a series of paired samples t-tests was conducted to analyse the between-text differences at the different language levels. As can be seen in Table 4.1, the observations made during the standard setting process were confirmed. 174 4 Results Language level Final score e-mail Final score story t df p Cohen’s d M SD M SD A1.1 and below 1.41 0.24 1.16 0.35 3.57 30 .001 0.64 A1.2 2.01 0.27 1.97 0.29 1.42 118 .158 0.13 A2.1 2.72 0.34 2.74 0.36 -0.48 148 .634 -0.04 A2.2 and higher 3.47 0.24 3.59 0.19 -2.12 20 .046 -0.46 Table 4.1 Results of paired samples t-tests examining between-text differences by language level While there was no significant difference between the mean final scores of the e-mail and the story at the levels A1.2 and A2.1, the lowest levels (A1.1 and below) showed a highly significant mean difference of 0.25, 95 % CI [0.11, 0.40], t(30) = 3.57, p = .001 with an effect size of d = 0.64, which is a medium effect size according to Cohen (1988). Thus, the learners at these levels, on average, performed significantly better in the e-mail task than in the story task. The learners at the highest levels (A2.2 and higher), on the other hand, showed a better performance, on average, in the story task, with a slightly significant mean difference of -0.12, 95 % CI [-0.25, -0.00], t(20) = -2.12, p = .046, and an effect size of d = -0.46, which is an almost medium effect size according to Cohen (1988). In addition to these results, it was observed that at levels A1.2 and A2.1 there was also a considerable number of profiles with between-text heterogeneity. At these levels, however, the number of learners who performed better in the e-mail (n = 128) was about equal to the number of learners who performed better in the story (n = 130). Figure 4.6 shows the between-text heterogeneity of the level A1.2 and A2.1 profiles. The distance between the lower and upper cut scores that define the beginning and the end of the language level is 0.77 for level A1.2 and 0.98 for level A2.1. Thus, an absolute between-text difference of 0-0.32 was considered as low, 0.33-0.64 as medium, 0.65-0.96 as high and >0.96 as very high between-text heterogeneity. As can be seen in Figure 4.6, slightly less than a third of the profiles at levels A1.2 and A2.1 showed a medium and 12 % of the profiles a high or very high between-text heterogeneity. 175 4.1 Part I: The young EFL learners’ writing competence Note. N = 268, M = 0.33, SD = 0.24 Figure 4.6 Between-text heterogeneity at levels A1.2 and A2.1 In summary, the analyses show that the learners at the lowest levels (A1.1 and below) performed significantly better in the e-mail task than in the story task, and that the learners at the highest levels (A2.2 and above) performed significantly better in the story task. Heterogeneity between the two tasks, however, was also found at the levels in-between (A1.2 and A2.1). While 56 % of the learners at these levels displayed a low between-text heterogeneity, 31 % showed a medium and 12 % a high or very high between-text heterogeneity. Important implications of these findings for classroom practice and research are discussed in chapter 5.1.5. 4.1.5 Communicative effect in young EFL learners’ narrative texts (RQ I.4) How do young EFL learners create a communicative effect in narrative texts? Since empirical findings on whether and how young EFL learners create a communicative effect in written texts are rare or do not exist (see chapter 2.8), the following chapter is devoted to a small qualitative analysis (n = 25) of this aspect in the learners’ narrative texts (story about the family meal, see Figure 3.6). 176 4 Results First, a statistical analysis was conducted to determine if there were signifi‐ cant differences in the scores for communicative effect / creativity between the different language levels. Since the data did not meet the requirements for a one-way ANOVA , a Kruskal-Wallis test was conducted. It was calculated with mean ranks, because the scores at the different language levels did not have the same shape, as assessed by visual inspection of a boxplot. Because of the small number of pupils at the levels below A1.1 (n = 5) and above A2.2 (n = 2), these two levels were subsumed under the levels A1.1 and below and A2.2 and above, respectively. Hence, the four levels A1.1 and below (n = 28+5, mean rank = 56.09), A1.2 (n = 119, mean rank = 126.01), A2.1 (n = 149, mean rank = 200.68) and A2.2 and above (n = 19+2, mean rank = 250.29) were used for the analysis. The test showed that the mean ranks of the scores for communicative effect / creativity were statistically significantly different between the different language levels, χ 2 (3) = 115.28, p < .001, with an effect size of η² = 0.35, which is a large effect size according to Cohen (1988). Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. Adjusted p-values are presented. This post hoc analysis revealed statistically significant differences in the scores for communicative effect / creativity between all language levels (p < .001), except for the combination A2.1 and A2.2 and above (p = .100), see Figure 4.7. Note. N tot = 322. n. s. = not significant. *p < .05. **p < .01. ***p < .001. Figure 4.7 Pairwise comparison of the scores for communicative effect / creativity at the different language levels 177 4.1 Part I: The young EFL learners’ writing competence Therefore, the subsequent qualitative analysis considered not only young EFL learners’ ability to create a communicative effect in narrative texts in general across language levels, but also the differences between the language levels. As can be seen in Table 4.2, the total number of elements used to create a communicative effect increases with language level. In the small sample used for this analysis, the texts below A1.1 contained none of these elements or up to two of them, at level A1.1 there were one to five elements per text, at level A1.2 three to seven, at level A2.1 three to twenty-one, and at level A2.2 seven to twenty-two elements. It seems important at this point to see these counts in the context of the previously discussed within-text heterogeneity (see chapter 4.1.4). The ratings for communicative effect/ creativity at level A2.1, for example, ranged from 0.0 to 4.0, and those at level A2.2 from 1.0 to 4.0. Hence the large differences in the number of elements at these levels. After this quantitative analysis, the chapter now turns to give a more descriptive account of how young EFL learners at different language levels create a communicative effect in narrative texts. Texts approaching level A1.1 were very simple in the way they created a communicative effect. While three texts did not contain any elements that triggered a communicative effect, there were two texts that contained very simple direct speech that caught the reader’s attention such as kom on däd (= Come on, Dad! ) or Weit iam koming (= Wait, I am coming.). One writer even created some tension when the son refused to follow his father to the kitchen: The vather went zum (= to) Fritz and sagte (= said) comon the ate. No im look the book. While the learners at level below A1.1 only wrote about that part of the story which was illustrated by pictures and did not add their own story endings, the majority of learners at levels A1.1 and A1.2 invented their own story endings (see Table 4.2, story ending). A learner at level A1.1, for example, wrote (grey font = words in the school language): Vahter was make ju ohh sas the vater Im finde the Book so spanend all go in the kitchen und have fun Die Tochter sas vather wen you Birthay bekom ju a book of my the vater Muther und tochter go in the Bed ‘Father, what make you? ’ ‘Ohh’, says the father, ‘I find the book so fascinating’. All go in the kitchen and have fun. The daughter says: ‘Father, when it’s your birthday, you receive a book from me’. The father, mother and daughter go in the bed. 178 4 Results Means Count a Examples <A1.1 A1.1 A1.2 A2.1 A2.2 Beginning of the story Direct start into the story 1 The good book. „Eating! ” says Mom. Papa Moll coms. But where was Filipp. Tension and relief Tension 1 3 4 6 She saw her father reading her book. „That’s enough! ”, she said, Culminating tension 1 „AAAA! ewer man do et alon! rief Mum. „Eating, eating, eating, Reduction or relief of tension 2 „What! go and say him et gaves eating! ” rief mum „Okay, Okay i go” means Filipp. Suspicious behaviour 1 Lisa went there and saw what her Father wose doing there: „Dad! What are you doing here? ” “Ahmm nothing! ” Unexpected elements Unexpected turn 1 4 11 10 Suddenly the mother get in the room. Narrated from the first-person perspective 1 My Dad and my mom waitet for me because it was ate time. Linguistic elements Direct speech that catches the reader’s attention 3 2 3 5 14 „How can a book be sow intresting? “, asked Norman. Expressions that catch the reader’s attention 2 13 Time flies by and dad was still not at the table. Words of emphasis 2 1 1 4 This book is very fat it has twothousand and twentyfor pages. 179 4.1 Part I: The young EFL learners’ writing competence Means Count a Examples <A1.1 A1.1 A1.2 A2.1 A2.2 Repetition of the same word 1 2 1 He's reading and reading and reading. Non-lexical conversation sounds 1 2 2 „Aghh dad come we have to eat! ” Textual elements Emphasis through adapted spelling 1 the Book was sooo fascinating Capitalisation of whole words 1 „AAAA! ewer man do et alon! Distinctive use of punctuation 5 1 The Mom waiting and waiting…… Drawings 2 Emotions Anger 1 2 She was wery angry. Surprise 1 1 „Dad! oh you lo[ok] at my book? Strongly expressed opinion 1 „Thats enough! “, she said, Humour Amusing element 3 „Dad! What are you doing here? ” “Ahmm nothing! ” „No dad i can see that you are reading my book about Barbies! ” „But it is very intrestin! ” 180 4 Results Means Count a Examples <A1.1 A1.1 A1.2 A2.1 A2.2 Interaction with the reader Phrase directed at the reader 3 3 So that was going very wrong. Question directed at the reader 2 2 Five minutes later Justin went upstairs too get his father. And what did she see? ! She saw her father reading her book. Story ending Own story ending 4 2 5 4 Ben found hes Mother and at the End the family readed the Book together! Witty story ending 1 1 Rolf and then Maria comes and they all read together. two ours leater Maia sais lets go and eat now. But when they are at the Table the food is gon they loock around and see theyr cat with a big tummy. Epilogue 1 1 That was the story of „the fascinating book”. Total number of elements 4 12 25 46 66 Note. a Count of the different text elements that create a communicative effect in the learners’ narrative texts. N = 25 (five randomly selected texts per language level that represent the distribution of scores for communicative effect/ creativity at each language level). Table 4.2 Means used by young EFL learners to create a communicative effect in narrative texts, differentiated by language level 181 4.1 Part I: The young EFL learners’ writing competence Or, in a story at level A1.2, Tim, the son, found his father who had fallen asleep while reading the book, and reported this to his mother: But dad don’t writing the book. He sleaping. Tim sead on mom: Daddy are sleaping! The mother left to bring the father back to the table, but did not return herself. So, eventually, Tim found the two fast asleep and ended up having dinner all by himself. These findings show that even at level A1, where the learners have rather limited language resources, a majority of learners appears to be able to invent and formulate an own story ending. One learner at level A1.2 added an epilogue to the story (That was the story of „the fascinating book”.), showing awareness of this particular genre-specific feature. Overall, the texts at the levels A1.1 and A1.2 contained about one to seven elements that created a small communicative effect. Some learners, for example, created some tension (The Mom waiting and waiting ......) or managed to include an unexpected turn in the story (Mom going to Dad and don’t coming return.). Additionally, and compared to the level below A1.1, they used more, and more varied, linguistic and textual elements to make the story interesting. The linguistic elements comprised direct speech that catches the reader’s attention (e. g. Daddy! Sed Tim it is my book.), words of emphasis (e. g. das Book is very cool), repetition of the same word (e. g. The Mom waiting and waiting ......) and non-lexical conversation sounds (e. g. ohh). The textual elements included emphasis through adapted spelling (e. g. the Book was sooo fascinating), distinc‐ tive use of punctuation (e. g. „Dad wat make you? ! ”) and drawings (e. g. ). Furthermore, a few learners expressed some emotions in the text such as anger (e. g. She was wery angry.) or surprise (e. g. ohh sas the vater = ‘Ohh’ says the father). At the language levels A2.1 and A2.2, the learners seem to have considerably more means at their disposal to create a communicative effect. Most texts at these levels showed that the learners were able to create some tension in a simple way (e. g. Lisa went there and saw what her Father wose doing there: „Dad! What are you doing here? ” “Ahmm nothing! ”) and include an unexpected turn (e. g. Dad come and eat. Okay answered Mr. Moll. But as they arrived at the table there was nobody there.). The learners at these levels used considerably more linguistic means to create a communicative effect (see Table 4.2, linguistic elements). The main linguistic elements were direct speech and expressions that caught the reader’s attention, e. g. „How can a book be sow intresting? “, asked Norman. Isabelle awnserd: „Well … be glad that your daughter is finaly reading! “ „Yes, your rigth“, said Norman, „but I'm still gonna bring her back to the dinningroom. After all she cant just stand up and walk away! “ or It was infact so intresting that she left her plate of spaghetti Nappoli just to go and read her book. Furthermore, the 182 4 Results learners used words of emphasis (At a very beautiful Saturday), repetition of the same word (He's reading and reading and reading.) and non-lexical conversation sounds („Aghh dad come we have to eat! ”). Some pupils described emotions such as anger (the son is going angry up[stairs]), surprise („Dad! oh you lo[ok] at my book? ), strongly expressed opinions („Thats enough! “, she said), or also incorporated humorous aspects into the text („Dad! What are you doing here? ” “Ahmm nothing! ” „No dad i can see that you are reading my book about Barbies! ” „But it is very intrestin! ”). A new element that did not appear at lower levels was the interaction with the reader. Some learners, for example, commented on the course of events in the story (So that was going very wrong. Everytime That one of them get to dinner one guy wasn't there. or They both knew who was missing. Their daughter of cource, Molly.), or raised questions that actively involved the reader in the events (And what did she see? ! She saw her father reading her book.). In one text, the readers were given information that was unknown to the characters in the story: Where was the father? The father reded the fascinating book and he can’t stoped. The mother sad to the child: „Go and look were your father is! ” The child goes to his room and he seys that his father was reding the fascinating book. At the levels A2.1 and A2.2, all analysed texts contained a story ending of the learner’s own. A few learners created an amusing story ending or added an epilogue. In summary, it can be concluded that young EFL learners with a writing competence at levels A1 and A2 already display some ability to create a com‐ municative effect in narrative texts in a simple way. The means by which such a communicative effect is created appear to increase and become more diverse with expanding language proficiency. Chapter 5.1.6 discusses the relevance of this aspect and draws some conclusions with regard to possible applications in the foreign language classroom. 4.1.6 Coherence in young EFL learners’ narrative texts (RQ I.5) What are the characteristics of coherence in young EFL learners’ narra‐ tive texts? This chapter presents the results of a small qualitative analysis (n = 25) of coherence in the narrative texts of young EFL learners. The reason for this analysis is that there do not appear to be any empirical findings on coherence in young EFL learners’ written texts as yet (see chapter 2.8.1). First, a statistical analysis was conducted to determine if there were signif‐ icant differences in the scores for coherence between the different language 183 4.1 Part I: The young EFL learners’ writing competence levels. Since the data did not meet the requirements for a one-way ANOVA , a Kruskal-Wallis test was conducted. It was calculated with mean ranks because the distributions of scores for coherence were not similar at the different language levels, as assessed by visual inspection of a boxplot. Because of the small number of pupils at the levels below A1.1 (n = 5) and above A2.2 (n = 2), these two levels were subsumed under the levels A1.1 and below and A2.2 and above, respectively. Hence, the four levels A1.1 and below (n = 28+5, mean rank = 50.29), A1.2 (n = 119, mean rank = 131.69), A2.1 (n = 149, mean rank = 195.15) and A2.2 and above (n = 19+2, mean rank = 266.38) were used for the analysis. The test showed that the mean ranks of the scores for coherence were statistically significantly different between the different language levels, χ 2 (3) = 114.91, p < .001, with an effect size of η² = 0.35, which is a large effect size according to Cohen (1988). Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. Adjusted p-values are presented. This post-hoc analysis revealed statistically significant differences in the scores for coherence between all language levels (p = .004 for the combination A2.1 and A2.2 and above, and p < .001 for all other combinations, see Figure 4.8). Note. N tot = 322. *p < .05. **p < .01. ***p < .001. Figure 4.8 Pairwise comparison of the scores for coherence at the different language levels 184 4 Results These findings suggest that there are significant differences in terms of co‐ herence between the different language levels, even at such a low level of competence. Therefore, the subsequent qualitative analysis not only considered coherence across the language levels, but also tried to pinpoint specific charac‐ teristics of coherence at each language level. The characteristics of coherence in young EFL learners’ narrative texts were analysed from three different perspectives. First, the texts were analysed with regard to the genre-specific structure of a simple narrative text. Based on Brinker et al. (2018) and Labov and Waletzky (1997), the elements setting of the scene, development, complication, resolution, story ending and evaluation / résumé were used as a framework for the analysis. According to Brinker et al. (2018), however, it is important to note that such a general thematic structure does not necessarily reflect the order in which a story is realised. For example, it may include several complications that occur at different points in the storyline, and which may be resolved successively or only at the very end. Thus, the analysis did not try to identify the specific location and number of these elements, but rather which elements were realised in the texts. As can be seen in Table 4.3, almost all texts included some sort of a development of the storyline. At the lowest level, however, only two texts contained a setting of the scene and a complication, and only one text had a story ending. Element Count a Examples <A1.1 A1.1 A1.2 A2.1 A2.2 Setting the scene 2 1 4 5 5 Theres wase once a family they wanted to have dinner but the child was mising Development 4 5 5 5 5 The mather asked the father: Wohr was Jackob? . Jack the father going and searched Jackob. Complication 2 1 3 3 4 Mom is very very Angry. Resolution 1 1 3 The fathe didn't want to go but he must. So he go. Story ending 1 3 3 4 5 That book is so fascinating and read the hole day. So they dindnt have lunch intil they finisched it. A long time later they were finished and hade lunch. Evaluation / résumé 3 And when the book isn't broke Robert and Peter are fighting till today for the book. Note. a Number of texts containing one or more elements of this type. N = 25 (five randomly selected texts per language level that represent the distribution of scores for coherence at each language level). Table 4.3 Elements of a simple narrative structure in young EFL learners’ narrative texts, differentiated by language level 185 4.1 Part I: The young EFL learners’ writing competence At level A1.1, three texts had a story ending but only one learner provided information about the setting of the scene, and only one text contained a complication. From level A1.2 onwards, the majority of texts, but not all, included a setting of the scene, a complication and a story ending. Interestingly, up to level A2.1 many of the analysed texts did not resolve the complications. Some of the learners ended the text at the very height of the complication (e.g. Mom and Jakop waiting of Dad beceas Dad is not her. Mom say: „go and look what Dad doing! “ Jakop walked to Dad and looked what Dad do. Dad riding the book from Jakop. Jakop liked this and riding wife hes Dad. Mom is very very Angry), and other learners continued the story as if there had not been any complication (e.g. And then the mom Matilda wase very mad so she went to get Ylli and Victor but then they were all reading the book. And they couldn't stop reading.). However, there were also a few texts that contained resolutions to the complications (e.g. His father came angry in the rom and say „Coming on the table son” and he thats making.). At level A2.2, three of the texts ended the story with a résumé or evaluation (e.g. And when the book isn't broke Robert and Peter are fighting till today for the book.). The second analysis concerned the number of gaps in the story. For this purpose, the one to three most important elements per picture were listed, so that missing elements could be easily identified. This resulted in a list of 18 elements for the whole story. As can be seen in Table 4.4, the average number of gaps decreased with language level, from 12.4 at level <A1.1 to 4.0 at level A2.2. There was one text at level <A1.1 that was almost completely written in German and, thus, could not be rated as including the key elements of the story (not enough assessable language). Language level Number of gaps in the story a M min max <A1.1 12.4 6 18 A1.1 9.8 7 13 A1.2 5.4 3 8 A2.1 5.6 2 11 A2.2 4.0 1 6 Note. a Gap = Missing element in the storyline. Based on an analysis of five randomly selected texts per language level that represent the distribution of scores for coherence at each language level. Table 4.4 Number of gaps in the story, differentiated by language level 186 4 Results This second analysis, however, does not provide any information about the quality of these gaps. Some gaps, for example, might strongly impair the understanding of the text, while others might not affect comprehensibility at all. For some gaps, the reader might be able to compensate by mentally adding missing elements, or other gaps may even have a positive effect, for example, by creating some tension. Thus, a third analysis was carried out that tried to describe the different types of gaps and incoherence at the different language levels, and their effect on the comprehension and overall coherence of the text. As can be seen in Table 4.5, the gaps observed at the two lowest levels often affected comprehension. They required the reader to read the text slowly and carefully, sometimes to reread it several times, while trying to mentally compensate for the missing elements. The first example in Table 4.5 is actually more concerned with punctuation than with coherence, but since the missing punctuation affects how well the reader is able to form units of meaning, it also concerns coherence. Type of gap or incoherence Example Level <A1.1 Missing punctuation hinders the reader in forming units of meaning. Comprehension im‐ paired. Were are Joschua gou and sie [=see] Joschuwa gou to de tabel wif te lanch wer ar däd go Joschua gou sie kom on däd koma n toe de lvingrum fud okei Joschua Weit iam koming okei Lack of context (e. g. setting of the scene) makes the text diffi‐ cult to understand. Storyline only comprehensible if pictures are provided. Were are Joschua gou and sie [=see] Joschuwa gou to de tabel wif te lanch wer ar däd go Joschua gou sie kom on däd koma n toe de lvingrum fud okei Joschua Weit iam koming okei Unclear who is speaking Were are Joschua gou and sie [=see] Joschuwa gou to de tabel wif te lanch wer ar däd go Joschua gou sie kom on däd koma n toe de lvingrum fud okei Joschua Weit iam koming okei Key information is missing, reader is puzzled about how the story continues. The vather went zum [= to] Fritz and sagte [= said] comon the ate. No im look the book. / / The Fritz went in the kitchen Complications are not resolved. Storyline does not seem to be complete. The Fritz went zum [= to] Vather vather comon the ate. No im look the book. [End of the story] Limited language resources (e. g. choice of a word that has a different meaning than that in‐ tended) result in a storyline that seems unclear. The mother quostion the Fritz who [= where] is the vather? Im look the book. 187 4.1 Part I: The young EFL learners’ writing competence Type of gap or incoherence Example Level A1.1 Lack of context (e. g. setting of the scene) makes it difficult to situate the events. [Beginning of the story] Where is my son? ” „I dont know. Go and look were he is. Unclear who is speaking Where is my son? ” „I dont know. Go and look were he is. Missing explanations make it dif‐ ficult to understand the actions or the reasons for these actions. Mr. Molly eats and goet in de romme [= room] fon [= of] is togter [= daughter]. She raitet [= read] a book. Hes togter goet eat. Mr. Molly raitet the book fon is togter. Abrupt change of the scenery makes it difficult to follow the text. Papa Moll woh [= where] is unser [= our] kid. I going out the House show kid. Kid why [= we] have many food coming in the house yes father. Ohh a intresing book. / / Mom who [where] is my father Level A1.2 Key information is missing, reader is puzzled about how the story continues. Jack the father going and searched Jackob. He not founded Jackob. / / Jackob going to the table / / but whow [= where] was the father. Abrupt change puzzles the reader. The mother says eating but the child does not comes then the father goes and holt the child / / the father looks at the book. Some information missing that adds tension to the story (the reader is not yet being told why the father is missing). Mom and Dad is in the kichen and waiting of shes children. Mom says: „go and look what he doing! “ Dad walked to Jakop and says: „what do you? “ Jakop says: „I riding a story“ „Okay beceas go to Mom and eated you breakefast! “ „Okay“ say Jakop. / / Mom and Jakop waiting of Dad beceas Dad is not her. Mom say: „go and look what Dad doing! “ Level A2.1 Incoherence The father dont find Timo. He was in his bedroom. Timo was reading a intresting book. The father said him he should go down to eat. Abrupt change puzzles the reader. Timo was looking were he is. The father was reading Timos book. / / Then the father must eat aloun. Parts of the original story left out. Creates a new storyline, which is nevertheless coherent. It was Sunday Morning the Mother (Beatrix) and the Father (Hank) wanted to eat Breakfast. But Fritz, the Child wasn’t there. Hank was going to look where Fritz was. He was reading a book. A fascinating book. Hank liked it to. / / So there readed the book together. When the Mother is comming in the room, she liked it to. So now Beatrix, Hank, an Fritz, are reading the book 188 4 Results Type of gap or incoherence Example Level A2.2 Unclear who is speaking It was not so logisch, but he explainded, that when I read he want to read to. „One time, I forgot to work, because I was reading! ” „Ahhh OK, nice story but can we eat? ”, I was very hungry. Only few elements missing that only slightly affect the coherence of the text. „Come on Peter“, said Robert „Mom did meat balls with soup“. Without giving Robert an answer he went to the eating table / / and now Dad Robert wasn't here. Note. Based on an analysis of five randomly selected texts per language level that represent the distribution of scores for coherence at each language level. Table 4.5 Types of gaps and incoherence in young EFL learners’ narrative texts, differentiated by language level At levels A1.2 and A2.1, the texts were comprehensible but still contained some gaps that puzzled the reader or required the reader to compensate for missing elements. Some of these texts, however, only contained few gaps that did not, or only slightly, affect the overall coherence of the text. At level A2.2, the analysed texts were coherent. Only rarely was there a gap that slightly affected the overall coherence of the text. Chapter 5.1.7 discusses the implications of these findings for teaching EFL writing to young learners. 189 4.1 Part I: The young EFL learners’ writing competence 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing 4.2.1 What teachers say about how they teach EFL writing at primary school (RQ II.1) What role do primary EFL teachers assign to writing compared to other skills and teaching components such as listening, speaking, reading, use of strategies, language and cultural awareness, spelling, grammar and vocabulary? The teachers were asked to indicate on a scale from 1 (very low) to 7 (very high) how much importance they assign to different teaching components in their grade six EFL classes. As can be seen in Table 4.6, the mean values of all teaching components are higher than 4.5. Therefore, none of the aspects, on average, is considered to be of low importance. Teaching component Average importance a n M SD Listening 19 6.68 0.58 Speaking 19 6.37 0.76 Vocabulary 19 6.00 0.88 Reading 19 5.95 1.08 Grammar 19 5.32 1.20 Language and cultural awareness 19 5.26 1.28 Spelling 19 5.16 1.34 Writing 19 5.05 0.91 Learning strategies 19 4.68 1.29 Note. a Rated on a scale from 1 (very low) to 7 (very high). Table 4.6 Importance assigned to different teaching components It can be observed that the oral skills (listening and speaking) are considered the most important, followed by vocabulary and reading. A low standard deviation of SD = 0.58 and 0.76 for listening and speaking indicates a comparably high rate 190 4 Results of agreement among the teachers about the importance of these aspects. Writing ranges at the lower end (M = 5.05, SD = 0.91), only followed by learning strategies with a mean importance of M = 4.68 ( SD = 1.29). Spelling is regarded, on average, as slightly more important than writing as a skill. The large standard deviation of SD = 1.34, however, indicates that the teachers attach varying degrees of importance to this aspect. Like spelling, the language resource grammar also ranks higher than writing as a skill. Chapter 5.2.1 discusses these findings in relation to the specifications of the cantonal curriculum ( BKS , 2018). (RQ II.2) What aims do primary EFL teachers pursue when teaching writing? The teachers’ most important aims when teaching EFL writing appear to be to consolidate vocabulary and to facilitate a positive writing experience, followed by communicating with somebody and consolidating sentences structures (see Table 4.7). Goal of teaching writing Average importance a n M SD Consolidating vocabulary 19 3.68 0.48 Facilitating a positive writing experience 19 3.63 0.50 Communicating with somebody 19 3.47 0.61 Consolidating sentence structures 19 3.32 0.58 Learning about new content 19 2.95 0.52 Consolidating orthography 19 2.89 0.46 Developing writing strategies 19 2.58 0.61 Learning about different genres 19 2.05 0.40 Practising handwriting 19 1.47 0.70 Note. a Rated on a scale from 1 (not important) to 4 (very important). Table 4.7 Importance assigned to different goals of teaching writing Again, the development of writing strategies, as well as learning about different genres, do not rank very prominently. Practising handwriting is regarded as the 191 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing least important among these aims. Given the opportunity to add further aims to the list, one teacher added that she wanted her learners to be able to give written feedback in English about the teaching lessons and to express their preferences and wishes. (RQ II.3) How frequently is text composition practised in class? As can be seen in Figure 4.9, about two thirds of the teachers get their learners to write short English texts once or twice a month, and longer English texts (more than half a DIN A4 page) once or twice per quarter year. Only a small number of teachers stated that they would not write either short or longer English texts, or that they would do so more frequently, for example once or twice a week. Note. N = 19. Figure 4.9 Frequency of text composition in grade 6 (RQ II.4) What aspects of text quality do the teachers discuss or address in class? According to the teachers, they most thoroughly address the aspects vocabulary, grammar and syntax in class (see Table 4.8). Cohesion, orthography and genre-specific aspects are dealt with less intensively. Level of detail, punctuation as well as aspects of the macro-level of text quality (coherence, communicative effect) are only dealt with rarely. 192 4 Results Aspect of text quality Average intensity a n M SD Vocabulary 19 1.68 0.48 Grammar (e. g. singular-plural, past tense) 19 1.68 0.48 Syntax (e. g. formulating question with “do you”) 19 1.42 0.61 Cohesion (e. g. linking words or personal pronouns) 19 1.21 0.54 Orthography 19 1.16 0.50 Genre-specific aspects (e. g. specific elements of a letter) 19 1.11 0.57 Level of detail (how to describe something in detail) 19 0.68 0.48 Punctuation 19 0.68 0.67 Coherence (how to structure a text) 19 0.58 0.51 Communicative effect (e. g. how to write a funny, sad or captivating story) 19 0.11 0.32 Note. a Rated on a scale from 0 (never), 1 (a bit) to 2 (thoroughly). Table 4.8 Aspects of text quality discussed or addressed in class (RQ II.5) What types of writing tasks and genres are used in class? In the questionnaire, the teachers were asked what types of written products the 6 th grade learners had produced in class during the past school year. According to the teachers, the most frequently produced texts were posters and presentations, stories, descriptions, recounts and letters (see Table 4.9 for some examples). While the genres factual texts (36 mentions), narratives (26 mentions) and correspondence (16 mentions) were used often, arguments (3 mentions) and dialogic texts (2 mentions) were implemented only rarely. Genre Count a Examples of writing tasks Factual texts 36 Posters / presentations 19 Volcanoes, famous painters, countries Descriptions 13 All about me, bizarre buildings, describing a painting 193 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Genre Count a Examples of writing tasks Procedures 3 How to prepare for a presentation, recipes Reports 1 Article about a baseball workshop Narratives 26 Stories 14 Retelling a story, Santa Claus, my life as a pharaoh Recounts 9 My last holiday, the long weekend, travel report Visions/ plans for the future 3 Plans for the weekend, my future Correspondence 16 Letters 9 Penfriend project with a class from another country E-mails 5 Responding to an e-mail Postcards 2 Postcard from Egypt Arguments 3 Flyers 2 How to save water Feedback to the teacher 1 Comments on the lessons, wishes for the les‐ sons Dialogic texts 2 Role-play 1 Self-written role-play about travelling Interview 1 Note. a As stated in the answers to the question ‘What kind of writing products were produced in class in the past year in grade 6? ’ Table 4.9 Types of genres and writing tasks implemented in class (RQ II.6) What types of pre-writing activities, scaffolding and feedback do the teachers employ when teaching EFL writing? Pre-writing activities As can be seen in Table 4.10, most teachers indicated that they cover the topic the pupils have to write about in class. They also seem to compile words and phrases the pupils can use while writing, and study sample texts. Collecting 194 4 Results ideas in class and discussing a possible text structure appear to be slightly less prominent. When asked to add further pre-writing activities they used in class, only one teacher reported that her pupils sometimes work in groups on a task before writing and use the internet, books etc. for this purpose. Pre-writing activity Average approval a n M SD The topic the pupils will write about is addressed in class. 19 3.42 0.90 We compile words and phrases the pupils can use. 19 3.16 1.01 We study a similar text in class as an example. 19 3.11 0.88 Together we collect ideas what the pupils could write about (e. g. mind-map). 19 2.84 0.90 In class, we discuss how the pupils can structure a text. 19 2.63 0.90 Note. a Rated on a scale from 1 (not true) to 4 (true). Table 4.10 Use of pre-writing activities in class Insights into the learners’ perception of pre-writing activities and some oral reports from the learner interviews can be found in chapter 4.2.2. Scaffolding Instructional scaffolding seems to be a central aspect when teaching EFL writing. All teachers fully agreed or agreed that the learners can look up words, ask the teacher for help and use text models during the writing process (see Table 4.11). The great majority of teachers also reported that the learners can ask classmates for help if they encounter a problem while writing. No further scaffolding techniques were added by the teachers. Type of scaffolding Average approval a Number of teachers n M SD Not true Rather not true Somewhat true True The pupils can look up words (e. g. in a dictionary or in the course book). 19 3.89 0.32 0 0 2 17 The pupils can ask me for help if they encounter a problem while writing. 19 3.84 0.37 0 0 3 16 195 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Type of scaffolding Average approval a Number of teachers The pupils can ask classmates for help if they encounter a problem while writing. 19 3.53 0.84 1 1 4 13 The pupils can use a similar text as a text model. 19 3.47 0.51 0 0 9 10 Note. a Rated on a scale from 1 (not true) to 4 (true). Table 4.11 Scaffolding during the writing process Feedback on the written product Similarly, the teachers reported that feedback is a commonly applied element of the writing process. All teachers agreed that the pupils can give them their texts to read through and most teachers reported that they give the learners feedback with regard to positive aspects and elements that need to be improved (see Table 4.12). One teacher additionally reported that she would not mark all mistakes but instead only focus on the most important ones. Many teachers also appear to give guidance during the revision process so that the learners know what aspects they should pay attention to when revising their texts. Type of feedback Average approval a Number of teachers n M SD Not true Rather not true Somewhat true True Pupils can give me their sentences and texts to read through. 19 3.89 0.32 0 0 2 17 I tell the pupils what they could improve (oral or written feedback). 19 3.68 0.58 0 1 4 14 I tell the pupils what is good (oral or written feedback). 19 3.63 0.60 0 1 5 13 During the revision process, I point out what the learners should pay attention to. 19 3.42 0.51 0 0 11 8 Note. a Rated on a scale from 1 (not true) to 4 (true). Table 4.12 Feedback on the written product In the learner interviews, some pupils described in detail how their teachers integrated feedback and revision in their writing lessons. This report can be found in chapter 4.2.2, where the learners’ perception of EFL writing is described. 196 4 Results (RQ II.7) What types of writing strategies are used in the classroom? In the following paragraph, a small selection of strategies has been selected for analysis. As can be seen in Table 4.13, strategies such as looking up words or asking someone for help seem to be the most prominent among the strategies listed. Further important strategies appear to be the use of text models, compiling words and sentences before writing and focusing on specific aspects when revising a text. Clearly, the strategies listed in Table 4.13 are only a small number of pre-selected questionnaire items, which cannot give a comprehensive overview of all strategies used in the classrooms. Neither does the available data provide answers to the question regarding the extent to which the strategies are explicitly taught, or rather used implicitly. Further research would be necessary if the use of writing strategies is to be examined more closely. Strategy Count a Looking up words while writing 17 Pupils can ask the teacher for help 16 Pupils can ask classmates for help 13 Using a text model 9 Compiling words and sentences before writing 9 Focusing on specific aspects when revising 8 Studying a sample text before writing 6 Considering genre-specific aspects of a text 4 Generating ideas before writing 4 Planning the structure of a text before writing 3 Note. a Number of teachers stating in the questionnaire that these elements are dealt with or used in their classes. Table 4.13 Writing strategies dealt with or used in the classroom While the data in Table 4.13 shows the teachers’ perception of the use of writing strategies in the classroom, chapter 4.2.2 additionally considers the learners’ perspective and reports on the difficulties they encounter while writing and the strategies they use to overcome them. 197 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing (RQ II.8) How do the teachers assess written products? In the teacher questionnaire, the teachers were asked (with an open question) what criteria they use to assess the learners’ written products. Criterion Number of teachers a Length 4 Communicative purpose Content 12 Comprehensibility 4 Creativity 3 Task completion 3 Use of genre-specific elements 1 Coherence and cohesion Coherence 7 Cohesion 4 Grammar and syntax Grammar 9 Syntax 7 Style 1 Vocabulary 10 Language mechanics Orthography 14 Copying correctly 3 Legibility 2 Effort 1 Note. a Number of teachers mentioning the crite‐ rion in their answer to the question ‘What criteria do you use to assess written products? ’ Table 4.14 Assessment criteria 198 4 Results As can be seen in Table 4.14, the most frequently mentioned criteria were orthography (14 teachers), content (12 teachers), vocabulary (10 teachers), grammar (9 teachers), coherence (7 teachers) and syntax (7 teachers). In general, it can be observed that most teachers assess in some way the extent to which the text fulfils its communicative purpose, as well as the quality of language mechanics. Further aspects of text quality such as vocabulary, grammar, syntax and coherence are only assessed by about half of the teachers, and cohesion was assessed by only four. Punctuation was not mentioned by any of the teachers. By categorising the different criteria into a macro level (communicative purpose, coherence), meso level (grammar, syntax, vocabulary, cohesion) and micro level of text quality (language mechanics), it was possible to observe that 13 teachers (68 %) mentioned criteria from all three levels, three teachers (16 %) from two different levels (either macro / meso or macro / micro) and one teacher (5 %) from one level (micro). Two teachers did not specify the criteria but stated that they would use a criteria sheet, or that the criteria would differ according to the task. 4.2.2 The pupils’ perception of EFL writing at primary school This chapter provides insights into the learners’ perception of EFL writing at primary school based on data from a questionnaire (n = 320) and learner interviews (n = 9). The aim of this chapter is to provide empirical findings that may help to deepen the understanding of the characteristics and challenges of EFL writing in a primary school context and to complement the findings presented in the previous chapters by giving the learners’ perspective. (RQ II.9) How much importance do the learners assign to learning to write in the English language? As can be seen in Figure 4.10, the great majority of learners perceives learning to write sentences and texts in English as important or quite important. 9 % regarded learning to write in the English language as not so important and 2 % regarded it as unimportant. 199 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Note. N = 320. Figure 4.10 Perceived importance of learning to write sentences and texts in English Two interesting statements were made by pupils during the interviews. One learner who did not like writing in English because he found it hard to find words and write something, nevertheless argued that it is important to learn to write in English: I: (…) findest du es gut dass ihr auf Eng‐ lisch schreibt? (…) do you think it’s good that you write in English? P: Ja ich glaube das ist schon wichtig weil Englisch ist eigentlich auch eine wichtige Sprache, ja. Yes I think that is important because English is actually an important lan‐ guage, yes. Another learner made a similar comment even though she did not like writing in English, since she often struggled with generating ideas and needed a lot of time, sometimes up to two hours or more, for writing a text: I: Und (..) findest du es eine gute Idee auf Englisch zu schreiben? And (..) do you think it is a good idea to write in English? P: Also ich finde man muss es schon können also darum würde ich mal sagen ja (lacht), wenn’s auch nicht immer Spass macht. Well I think you have to be able to do it, well that’s why I would say yes (laughing), even if it is not always fun. I: Und es macht dir nicht Spass, oder nicht immer Spass, [weil du lange brauchst? ] And you’re not having fun, or not always having fun, because it takes you so much time? 200 4 Results P: Nicht immer Spass. Ja. Ziemlich lange. Not always fun. Yes. Quite long. Unfortunately, the interview data did not provide any information from learners who found learning to write unimportant or not so important. Thus, no information can be provided about the reasons for their opinion. (RQ II.10) How much confidence do the learners have in their ability to write in English? As can be seen in Figure 4.11, about 42 % of the learners reported that they have a high confidence in their ability to write in English (self-efficacy). 52 % of the pupils indicated a medium and 6 % a low confidence in their ability to write in English. Note. N = 320. Measured with a scale consisting of three items (see Appendix C, Table C1), ranging from 1 (not true) to 4 (true). Low ≤ 2.0, medium > 2.0 and ≤ 3.0, high > 3.0. Figure 4.11 The learners' confidence in their ability to write in English A statistical analysis was conducted in order to investigate whether there was a significant difference between the learners’ self-efficacy in writing in English and their general self-efficacy in learning English. Since the data did not meet the requirements for a paired-samples t-test, a Wilcoxon Signed-Ranks Test was conducted. The test showed that the learners’ self-efficacy in learning English (Mdn = 3.5) was significantly higher than their self-efficacy in EFL writing (Mdn 201 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing = 3.0), Z = -12.7, p < .001, with an effect size of r = 0.71, which is a large effect size according to Cohen (1988). (RQ II.11) What do the learners like and dislike about writing in English? In the interviews, the learners were asked what they liked and disliked about writing in the foreign language, and why. The pupils mentioned various aspects that contributed to their perception of EFL writing: their general attitude and motivation, the type of tasks, the availability of resources and the purpose of getting information about the current stage of learning. General attitude and motivation According to the learners participating in the interviews, the general attitude towards languages (e. g. I very much like writing in English because I think languages are great) as well as the learners’ motivation appear to affect whether they like or dislike writing in English. One pupil, for example, explained: P: Also manchmal mache ich’s gerne, wenn ich wirklich die Motivation habe und wenn ich dann Null Motivation habe dann nicht. Well, sometimes I like doing it, if I am really motivated, but if I am not motivated at all, then I don’t. I: Und was gibt dir Motivation? And what gives you motivation? P: Hm (.....) wenn ich einen guten Tag habe ja. Wenn bis jetzt alles gut ge‐ laufen ist und, (.) dann mache ich das auch gerne. Well (.....) if I have a good day, yes. If everything went well so far and, (.) then I like doing it. Task types Furthermore, a great number of aspects referring to task types were mentioned that appear to affect the learners’ perception of EFL writing. However, the learners did not always agree on what they liked and disliked. Some learners reported that they like EFL writing because they can invent something new (e. g. you can always invent new things) and express their ideas (e. g. because I had such a funny idea). One learner said that she liked stories best where she is free to write whatever she likes. Other learners, however, found it difficult to generate ideas (e. g. what I did not find easy is to come up with it). This learner preferred the picture story because the main storyline was already given: 202 4 Results I: Was hast du lieber gemacht? Das E-Mail schreiben oder die andere Ge‐ schichte fertig schreiben? What did you prefer? Writing the e-mail or finishing the other story? P: Ich fand die andere Geschichte besser (..) weil- I preferred the other story (..) because- I: Warum? Why? P: Hmmm. Weil da schon was vorge‐ geben war und man da nicht so lange sich Gedanken zu machen musste. Um. Because something was already given and you did not have to think about it for so long. For this learner, the length of the text also appears to play a role. He reported that he does not like writing long texts (… where you have to write something big, a lot. And in English it is sometimes a bit hard for me to write in length, I mean, a lot). He preferred short stories, no longer than half a page. A girl liked texts best where she could additionally draw something or decorate the border of the text. Another girl did not like writing texts in test situations because she experienced them as stressful, the time was usually limited, and she feared that she might not come up with the necessary ideas: I: Und was am wenigsten gern? And what [did you like] the least? P: Ähm, am wenigsten gern, die Texte wo wir in den Prüfungen hatten. Um, the least, the texts we had in the exams. I: Warum? Why? P: Weil das hat, man hat ein bisschen Stress weil es ja eine Prüfung ist und man hat nicht so viele Ideen weil man nicht so viel Zeit mehr hat und des‐ wegen kommen nicht so viele Ideen, da muss man wirklich so, ja. Because there is, you have a bit of stress because it is an exam, and you don’t have so many ideas because you don’t have much time left and that’s why not so many ideas come up, then you really have to, like, yes. I: Schnell, schnell schreiben. Quickly, write quickly. Availability of resources Moreover, the learners’ likes and dislikes seem to be closely linked to the resources available for writing. One frequently mentioned aspect was the availability of vocabulary resources. On the one hand, there were learners who stated that they feel confident when writing in English because they already know many words. One learner, for example, argued that she liked writing in the foreign language because she can use the words she has learned. On the 203 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing other hand, several learners argued that they did not like writing in English because of their limited vocabulary resources: I: Machst du das gerne? Do you like doing this? P: (lacht) mh eigentlich nicht, es fällt mir manchmal wirklich schwer ir‐ gendwelche Wörter zu finden und auch wirklich was zu schreiben (laughing) um actually not, it is some‐ times really hard for me to find any words and to actually write something In addition, having or not having ideas appears to affect whether the children like or dislike EFL writing. One learner liked writing a story in English because she had an amusing idea: I: Hast du das gerne gemacht? Did you like doing this? P: Ja! Das habe ich am liebsten gemacht. Yes! This was my favourite thing to do. I: Die zweite Geschichte vor allem? The second story mainly? P: Ja. Yes. I: Warum hast du das so gern gemacht? Why did you like it so much? P: Weil das, weil ich so eine lustige Idee hatte und halt einfach losschreiben konnte. Because this, because I had such an amusing idea and could just directly start writing. On the other hand, not having ideas or not having enough time to generate ideas (e. g. in a test situation, as mentioned above) appears to be stressful for the learners. The time factor was also mentioned by a girl who did not like writing in English because she usually needed a lot of time to write a good text. The purpose of getting information about the current stage of learning Lastly, one learner thought it was good to write the texts for this study to get information about their current stage of learning. (RQ II.12) What topics would the learners like to write about and why? When asked what they would like to write about, the learners mentioned a wide range of topics. The great majority of learners who were interviewed preferred to write narratives and personal recounts, e. g. an own story or texts about their 204 4 Results school, holidays or leisure time. One boy had a very clear idea of what he wanted to write about: I: Wenn du selber wählen könntest, wo‐ rüber würdest du schreiben? If you could choose for yourself, what would you write about? P: Ähm über eine Geschichte mit einem Jungen der, der, der auf, der neu in eine Stadt kommt, und sich da zuerst einleben muss und dann er so tolle Abenteuer erlebt und was Neues en‐ tdeckt und so. Um, about a story with a boy who, who, who is new to a city, and who has to settle in first and then he ex‐ periences such great adventures and discovers new things and so. One learner wanted to write about her class ‘because our class is a bit crazy’, her holidays or her personal future, and another girl felt like writing an autobiography (I’d like to write my life in English). Some learners preferred to write factual texts, e. g. about different sports, and one learner had a pen friend in England with whom she loved to exchange letters. Another learner wanted to translate an English book because he had spotted many mistakes in an existing German translation. The learners also commented on the reasons why they would like to write such texts. Having something interesting to tell, creativity (e. g. inventing new things), interest in the topic (e. g. sports), interest in text design (adding complementary drawings or decoration to a text) and the idea of improving an existing translation were mentioned by the pupils. The learners, however, also gave pragmatic reasons for their preferences such as ‘That’s the easiest’ or having enough ideas: I: Wenn du selber wählen könntest, wo‐ rüber würdest du schreiben? If you could choose for yourself, what would you write about? P: Ähm wenn ich selber wählen könnte würde ich über meine Freizeit schreiben oder das was ich mache, weil da würde es mir am leichtesten fallen, weil ich da genug Ideen hätte. Um, if I could choose for myself I would write about my spare time or what I do, because it would be easiest for me, because I would have enough ideas. When the learners gave reasons for dislikes, it was mainly due to difficulties during the writing process, e. g. difficulties in finding words, not having enough ideas, time pressure, great expenditure of time or having to write a lot. 205 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing (RQ II.13) What procedures do the learners experience when they compose texts in class? Pre-writing activities Both the learner questionnaire and the interviews explored the pupils’ percep‐ tion of pre-writing activities in the EFL classroom. As can be seen in Table 4.15, addressing the topic in class was reported to be the most frequently used pre-writing activity (M = 3.31 on a scale ranging from 1 to 4), followed by compiling words and phrases that might be useful for the writing task (M = 2.79). Collecting ideas in class was reported to be the least frequently applied pre-writing activity (M = 2.33). Only one class gave a high approval mark to this aspect. Pre-writing activity Average approval a Number of classes with low, medium and high approval b n M SD Low Medium High We address the topic in class. 319 3.31 0.77 0 0 19 We compile words and phrases. 318 2.79 0.92 0 9 10 We discuss a possible text struc‐ ture. 320 2.57 0.89 2 12 5 We study a sample text. 318 2.44 1.02 4 10 5 We collect ideas in class. 320 2.33 0.93 7 11 1 Note. a Rated on a scale from 1 (not true) to 4 (true). b Level of approval: Percentage of learners per class who agree with the statement. Low approval: ≤ 35 %, medium approval: > 35 % and ≤ 65 %, high approval: > 65 %. Table 4.15 Pre-writing activities: What do you do before you write a text in English? In the interviews, one learner gave an example of how they discussed the text structure: I: Wenn ihr im Englischunterricht einen Text schreibt. Wie geht ihr dann vor? Also was macht ihr in der Klasse bevor ihr ihn schreibt? When you write a text in an English lesson. How do you go about it? I mean, what do you do in class before you write it? 206 4 Results P: Wir besprechen zuerst was alles muss stehen und was alles im Text muss sein und dann mit den Abläufen dass die Einleitung zuerst kommt und dann es immer spannender wird und es dann eher wieder am Ende (...) wieder ähm wieder sich die Spannung schliesst und ja. We first discuss what has to be in‐ cluded and what has to be in the text and then the procedure, that the intro‐ duction comes first and then there is more and more tension and then at the end it rather (...) again, um, the tension closes again and yes. Another learner reported about the use of sample texts: I: Wenn ihr im Englischunterricht eine Geschichte schreibt, einen Text, was macht ihr in der Klasse bevor ihr den Text schreibt? Wie geht ihr vor, also überlegt ihr gemeinsam worüber ihr schreibt? When you write a story in an English lesson, a text, what do you do in class before you write the text? How do you go about it, I mean, do you discuss together what to write about? P: Ähm nein eigentlich immer selber (...) also Um, no actually always by ourselves (...) I mean I: Äh schaut ihr einen Beispieltext an? Uh, do you study a sample text? P: Ja, eigentlich meistens. Wir mussten auch schon mehrere E-Mails schreiben, dann haben wir im Pupil’s Book immer ein Beispiel gehabt und dann haben wir das alle zusammen angeschaut und nachher mussten wir selber eines schreiben. Yes, actually most of the time. We already had to write several e-mails, then we always had an example in the Pupil’s Book and then we all looked at it together and afterwards we had to write one ourselves. Scaffolding during the writing process The study also investigated the learners’ perception of different types of instructional scaffolding available during the writing process, namely the use of text models and dictionaries as well as opportunities to ask the teacher or peers for help. As can be seen in Table 4.16, almost all learners stated that they can ask their teacher for help. Three quarters of the learners indicated that they have the opportunity to ask other children for help and to look up words. The use of a text model seems to be less common. 43 % of the learners reported that they can use a similar text as a template for writing their own texts. 207 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Learners agreeing N tot n % If I have a problem, I can ask the teacher for help. 319 313 97.8 If I have a problem, I can ask other children for help. 320 247 77.4 While writing, we can look up words (e. g. in a dictionary or in the course book). 319 243 75.9 While writing, we can use a similar text as a text model. 320 138 43.3 Table 4.16 The learners’ perception of scaffolding during the writing process In the interviews, the learners provided additional details on what tools they use for looking up words. Most learners reported that both print and digital tools were available in their classrooms. They used print books such as their course books or dictionaries, as well as digital applications on computers, online dictionaries, online translation tools or a language translator device. Feedback and revision According to the learners’ reports in the interviews, there appear to be diverse ways how the teachers deal with feedback and revision. In one class, the children reported that they would first reread their texts themselves and then give it to the teacher for proofreading. Sometimes they made corrections and sometimes they left the texts as they were. In another class, the teacher similarly asked the children to read their texts themselves before she proofread them. She gave written feedback and wrote down some key words the children then had to learn. The children corrected their texts and added the highlighted words to a personal learning list. In the written feedback, the teacher referred to positive aspects and elements to be improved: P: ähm, sie schreibt immer etwas Kleines dazu zum Beispiel, ähm, der Text ist sehr spannend ähm pass nächstes Mal auf dass du die Verben richtig kon‐ jugierst und dann schreibt sie auch Wörter auf die dumme Fehler waren und die kommen dann in ein solches Heft, wo wir alle Wörter sammeln, ja, die wir halt eben noch nicht so gut sind und so. um, she always writes something small to it for example, um, the text is very exciting, um, make sure you con‐ jugate the verbs correctly next time and then she also writes words down which were silly mistakes and then they go into such a notebook where we collect all words, yes, which are, well, not so good yet and so on. 208 4 Results Another teacher was reported to first proofread the children’s drafts, give additional oral feedback and then have the children write their final products: P: Ja also wir schreiben zuerst auf Häus‐ chenpapier oder Sudelpapier, dann gehen wir zur Lehrerin, sie korrigiert das dann übers Wochenende oder am Abend, gibt es am nächsten Tag uns zurück mit den Fehlern und besprecht es mit uns kurz dann können wir es reinschreiben Yes, well, we first write on squared paper or notepaper, then we go to the teacher, she then proofreads it over the weekend or in the evening, gives it back to us the next day with the mis‐ takes and discusses it with us briefly then we can write it neatly Sometimes, if a child did not do very well, the teacher discussed the text personally with him or her: P: Ähm, wir geben es der Lehrerin ab, sie korrigiert es dann. Ähm, wenn es eine Prüfung ist, gibt sie es uns zurück und wenn wir’s ganz, also gar nicht gut gemacht habt, haben, ähm, schaut sie es mit einem an, also sie besprecht es und fragt warum ist es so und so passiert, und dann kann man immer reden darüber und beim nächsten Mal klappt es dann. Um, we give it to the teacher, she then proofreads it. Um, if it is a test, she gives it back to us and if we have done it very, I mean, not well at all, um, she reviews it with you, I mean she discusses it and asks why it happened like this or that, and then you can always talk about it and the next time, it works well. Audience and publishing According to the learners, their English teachers are the main audience who reads the texts they have written in English, followed by their parents and classmates. In addition, other people at school (e. g. children and teachers from other classes), siblings, friends and other relatives (e. g. grandparents or cousins) sometimes read their texts. When asked in the interviews what they do with their final written products, most learners reported that they store them in a file, folder or notebook, or hang them up in the classroom. Sometimes they read each other’s texts or use them for a class activity (e. g. someone reads a text and the others have to guess who the author is). (RQ II.14) What difficulties do the learners encounter when writing in English and what strategies do they use to overcome them? 209 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Difficulties encountered when writing in the foreign language Table 4.17 gives an overview of the different types of difficulties the learners mentioned in the interviews. Besides difficulties with regard to text quantity, idea generation, predetermined content and genre-specific expectations, the learners mainly mentioned different linguistic difficulties. Difficulty encountered Example quote (translated from German) Text quantity “And in English it is sometimes a bit hard for me to write in length, I mean, a lot.” Finding ideas “What I didn’t find easy was making it up.” Predetermined content “Um, that it was already prescribed and then to continue writing with these things, for example with the family meal, um there were, I think, pictures and yes I found that a bit difficult.” Genre-specific elements “Well, you had to think carefully how, because you can’t write just anything when you have to write an e-mail, you have to think a bit about what is expected in an e-mail, or what you usually write, yes.” Vocabulary “If I don’t know the words, then it gets difficult.” Sentence structures “Um (…) not so much because it is actually quite difficult for me to translate words from English, I mean, from German into English and to rearrange the sentences so that it fits into English, yes.” Tenses “Um it’s difficult when we have to write in the past tense or something like that because sometimes, sometimes I’m not quite sure any more what the rule is or the verbs are irregular and you have to know them by heart and that’s a bit arduous.” Orthography “But the spelling, the (.) is sometimes not so good, I mean, I can’t always write it correctly.” Mixing of languages “But I sometimes can’t […] write them correctly, because I think in Englit also mixes a bit with French, maybe there’s an ‘e’ at the end, which then maybe isn’t correct either.” Table 4.17 Difficulties encountered during the writing process The availability or absence of language resources also appears to affect non-lin‐ guistic aspects such as the content or the generation of ideas. As can be seen in the following extract, the learners may well have ideas on what to write about, but lack the necessary language resources to express them: 210 4 Results I: Ja, fällt es dir leicht die Ideen zu finden? Yes, is it easy for you to find the ideas? P: Hmm, nein, nicht wirklich also, es fällt mir in Deutsch irgendwie leichter weil da kann ich irgendwie, weil ich kenn da auch mehr spezielle Wörter die ich im Englischen gar nicht kenne deswegen fällt’s mir leichter, Um, no, not really, well, it is somehow easier for me in German because there I can, somehow, because there I also know more special words which I don’t know in English, that’s why it’s easier for me, I: Auf Deutsch dann, aber die Idee hät‐ test du? In German then, but the idea you’d have? P: Ja. (lacht) Yes. (laughing) Therefore, limitations in terms of content or communicative effect might not only be due to limited world knowledge, or be the result of a learner lacking good ideas. The learners may simply not have the necessary language to express what they would like to write about. The following learner describes how he sometimes struggles to find the right words and finally gives up his search and decides to write about something else: I: Ähm, was machst du denn wenn du die Wörter nicht kennst? Um, what do you do if you don't know the words? P: Eigentlich frage ich nach oder, bei der Lehrerin oder ich, eigentlich frage ich zuerst meine Pultnachbarn dann meine Lehrerin und (…) vielleicht nochmal in einem Wörterbuch aber wenn ich dann nichts weiss dann (…) brauche ich vielleicht eine andere Idee oder sowas. Actually I ask for it or, the teacher or I actually ask my desk neighbours first, then my teacher and (…) maybe again in a dictionary but if I then still don’t know anything then (…) maybe I need another idea or something. Strategies used to overcome difficulties According to the interview data, the learners appear to use a wide variety of strategies to overcome these difficulties. Most frequently, the learners reported that they look up words or ask the teacher for help (see Figure 4.12). Further common strategies seem to be paraphrasing, trying to find another word, or writing the missing word in the school language. 211 4.2 Part II: Current teaching practices and the learners’ perception of EFL writing Note. N = 9. Figure 4.12 Strategies to overcome the difficulty of missing vocabulary (frequency of their being mentioned in the interviews) Two learners, however, also reported that they were not always successful when trying to express their ideas in other words (well, I sometimes try but somehow it doesn’t work out for me). When confronted with the challenge of finding good ideas, the learners seem to ask the teacher or peers for help, look around in the classroom (Yeah, I kind of look around the classroom and maybe I see something -laughs-) or just think more about it. In order to overcome spelling difficulties, the learners reported using strategies such as trying to write the words as correctly as possible, looking them up or trying to remember what they had learnt in class. Furthermore, one learner reported that they would use an online translation tool if they were not able to figure out certain sentence structures. 4.3 Part III: Predictors of the learners’ EFL writing competence 4.3.1 Individual factors (RQ III.1) Is the self-efficacy of young EFL learners a predictor of their EFL writing competence? 212 4 Results In order to answer this question, the concept of self-efficacy was divided into two aspects: self-efficacy in writing in English, and the more general self-efficacy in learning English. First, a linear regression was run to understand the effect of the learners’ self-efficacy in writing in English on their EFL writing competence. To assess linearity, a scatterplot of the learners’ writing competence against their self-efficacy in writing in English was plotted. Visual inspection of this scatterplot indicated a linear relationship between the variables. There was homoscedasticity and normality of the residuals, and there were no outliers. The prediction equation was: EFL writing competence = 0.568 + 0.592*self-efficacy in writing in English (see Figure 4.13). Figure 4.13 Scatterplot of EFL writing competence against self-efficacy in writing in English with superimposed regression line The learners’ self-efficacy in writing in English was a statistically significant predictor of their writing competence, with β = 0.55 (p < .001) and an overall model fit of F(1, 318) = 137.48, p < .001. The learners’ self-efficacy in writing in English accounted for 30.0 % of the variation in EFL writing competence (adjusted R 2 ), which is a large effect size (f = 0.65) according to Cohen (1988). A second linear regression was run to analyse the effect of the learners’ general self-efficacy in learning English on their EFL writing competence. To assess linearity, a scatterplot of the learners’ writing competence against their self-efficacy in learning English was plotted. Visual inspection of this scatterplot indicated a linear relationship between the variables. There was 213 4.3 Part III: Predictors of the learners’ EFL writing competence homoscedasticity and normality of the residuals, and there were no outliers. The prediction equation was: EFL writing competence = 0.223 + 0.623*self-efficacy in learning English (see Figure 4.14). The learners’ self-efficacy in learning English was a statistically significant predictor of their EFL writing competence, with β = 0.49 (p < .001) and an overall model fit of F(1, 318) = 102.27, p < .001. The learners’ self-efficacy in learning English accounted for 24.1 % of the variation in EFL writing competence (adjusted R 2 ), which is a large effect size (f = 0.56) according to Cohen (1988). Figure 4.14 Scatterplot of EFL writing competence against self-efficacy in learning English with superimposed regression line Thus, both the learners’ self-efficacy in writing in English and their general self-efficacy in learning English were statistically significant predictors of their EFL writing competence. The strength of this association remained the same in a multiple regression analysis. A multiple regression model that included both self-efficacy variables statistically significantly predicted the learners’ EFL writing competence, F(2, 317) = 77.30, p < .001, accounting for 32.4 % of the variation in EFL writing competence (adjusted R 2 ), which is a large effect size (f = 0.69) according to Cohen (1988). Both variables added statistically significantly to the regression model, p < .001. The regression coefficients, confidence intervals and standard errors are shown in Table 4.18. 214 4 Results EFL writing competence B 95 % CI for B SE B β R 2 Adj. R 2 LL UL Model .33 .32 *** Constant 0.10 -0.30 0.50 0.20 Self-efficacy in writing in English 0.43 *** 0.30 0.56 0.07 0.40 *** Self-efficacy in learning English 0.28 *** 0.12 0.44 0.08 0.22 *** Note. Model = “Enter” method in SPSS Statistics; B = unstandardised regression coefficient; CI = confidence interval; LL = lower limit; UL = upper limit; SE B = standard error of the coefficient; β = standardised coefficient; R 2 = coefficient of determination; adj. R 2 = adjusted R 2 . * p < .05. ** p < .01. *** p < .001. Table 4.18 Multiple regression model for the learners’ self-efficacy as predictor of their EFL writing competence (RQ III.2) Is the young EFL learners’ extra-curricular use of English a predictor of their EFL writing competence? In order to analyse the effect of the young EFL learners’ extra-curricular use of English on their EFL writing competence, a linear regression was conducted. To assess linearity, a scatterplot of the learners’ EFL writing competence against their extra-curricular use of English was plotted. Visual inspection of this scatterplot indicated a linear relationship between the variables. There was homoscedasticity and normality of the residuals and there were no outliers. The prediction equation of the regression analysis was: EFL writing competence = 1.734 + 0.362*extra-curricular use of English (see Figure 4.15). 215 4.3 Part III: Predictors of the learners’ EFL writing competence Figure 4.15 Scatterplot of EFL writing competence against extracurricular use of English with superimposed regression line The learners’ extra-curricular use of English was a statistically significant predictor of their EFL writing competence, with β = 0.44 (p < .001) and an overall model fit of F(1, 318) = 74.07, p < .001, accounting for 18.6 % of the variation in EFL writing competence (adjusted R 2 ), which is a large effect size (f = 0.48) according to Cohen (1988). 4.3.2 Educational factors (RQ III.3) Are teaching factors such as the frequency of text composition in class, the importance assigned to reading, the frequency of vocabulary learning, and the role assigned to pre-writing activities, instructional scaffolding, feedback and orthography predictors of the learners’ EFL writing competence? Since the data to answer this research question had a multilevel structure (pupils nested within classes, see Figure 4.16), a multilevel analysis was conducted in order to examine the effect of different teaching factors on the learners’ EFL writing competence. 216 4 Results Figure 4.16 2-level hierarchical structure of the multilevel analysis As discussed in chapters 3.4.4 and 3.6.4, the learners’ responses to questions about classroom instruction were not reliable enough to be used for the analysis. Therefore, it was only possible to use the teachers’ responses (n = 19) for this purpose, which considerably limited the options and the power of the analysis. The small sample size did not allow for more than one or two variables to be included in a model, the chances of obtaining significant results were low, and the danger of type II error increased. While the risk of type II error could be dealt with by selecting a different type of estimation procedure ( RIGLS / REML , see McNeish, 2017), the other limitations remained. Therefore, only the results of the null model are presented, which provides information on how much variance is to be attributed to the class level, and some simple random intercepts models that might provide some indication for further research. To assess linearity, scatterplots of the learners’ EFL writing competence against the different independent variables were plotted. Visual inspection of these plots indicated a linear relationship between the variables. There was homoscedasticity, as assessed by visual inspection of catch-all plots of the standardised level-1 residuals against the fixed part predictions. Residuals were approximately normally distributed as assessed by visual inspection of normal probability plots. In none of the models were there outliers at class level, as assessed by examination of studentised residuals for values greater than ±2 (Rasbash, Steele, Browne, & Goldstein, 2020). There were three outliers at level 1, which did not appear to be data entry or measurement errors and were thus considered as genuinely unusual data points. The analyses were run twice, once with and once without the outliers. Since the goodness of fit statistics were similar, the outliers were not removed. Model 1: Null model First, the null model was calculated in order to estimate how much variance was to be attributed to the class level (see Figure 4.17). 217 4.3 Part III: Predictors of the learners’ EFL writing competence Figure 4.17 Model 1: Null model As can be seen in Table 4.19, 12.7 % of the total variance was associated with the class level. Thus, the use of multilevel modelling appeared to be warranted (see Heck, Thomas, & Tabata, 2010). Model 2: Random intercepts model with one level 2 predictor It was assumed that a considerable amount of this class level variance was influenced by different proportions of highand low-achieving pupils in the classes. A class with a large number of high-achieving pupils, for example, was expected to have a higher average writing competence than a class with a large proportion of low-achieving pupils. Thus, the proportion of highand low-achieving pupils (operationalised by the grand-mean-centred class average of the learners’ overall mark in the school subject English) was added to the model as a level 2 predictor (see Figure 4.18). Figure 4.18 Model 2: Random intercepts model with one level 2 predictor 218 4 Results Table 4.19 Results of the multilevel analysis 219 4.3 Part III: Predictors of the learners’ EFL writing competence As can be seen in Table 4.19, model 2 fits the data significantly better than the null model (p = 0.010). When controlling for the proportion of highand low-achieving pupils in the classes, the class level variance decreased from 12.7 % in the null model to 8.0 % in model 2. In other words, 4.7 percentage points of the class level variance could be traced back to different proportions of highand low-achieving pupils in the classes, and 8.0 percentage points remained unexplained. Models 3.1 - 3.7: Random intercepts models with two level 2 predictors On the basis of model 2, further random intercepts models with two level 2 predictors were calculated in order to investigate whether it was possible to identify an influence of different educational factors on the learners’ EFL writing competence (see Figure 4.19). Due to the small sample size on the class level (n = 19), the different educational factors could not be added jointly to the same model, but a separate model had to be calculated for each variable. All level 2 predictors were grand-mean-centred. Figure 4.19 Model 3: Random intercepts model with two level 2 predictors As can be seen in Table 4.19, none of the models that included an educational factor fits the data significantly better than model 2 (p > .05). While models 3.2-3.7 were clearly non-significant, model 3.1, which examined the effect of the frequency of text composition in class, was only slightly above the significance level of p = .05, and the unexplained class level variance was reduced from 8.0 % to 6.2 %. This result might indicate that the frequency of text composition in class could have an effect on the learners’ EFL writing competence, but clearly further research is needed to confirm this assumption. For further illustration, Figure 4.20 shows the model 3.1 predictions. While there seems to be a certain tendency towards a higher EFL writing competence with more text composition practice, it also becomes clear that a similar frequency of text composition practice does not automatically seem to lead to a similar average EFL writing competence in the different classes (see e. g. frequency 2.5 in Figure 4.20). This appears to be plausible, considering how 220 4 Results many different factors at the class level there are which could have an influence on the learners’ EFL writing competence (see chapter 2.2.3). Note. N = 19. For descriptive data on how frequently text composition was practised in the classes, see chapter 4.2.1. Figure 4.20 Model 3.1 predictions In summary, the research question III .3 tried to address the influence of different educational factors on the learners’ EFL writing competence. The available data did not allow final conclusions to be drawn, but the analysis nevertheless revealed some noteworthy findings. First, the analysis showed that 12.7 % of the total variance could be attributed to the class level, with 4.7 percentage points being the result of differences in the proportion of highand low-achieving pupils in the classes. Regarding the different educational factors, none of them could be shown to significantly contribute to the explanation of the differences in the learners’ EFL writing competence, which may, inter alia, be due to the limited sample size on class level. The frequency of text composition in class, though, showed results which come close to being significant, indicating that it may be a factor worth further investigation. 221 4.3 Part III: Predictors of the learners’ EFL writing competence 5 Discussion 5.1 Part I: The young EFL learners’ writing competence The aim of Part I of this study was to give a detailed account of the young EFL learners’ writing competence in grade six at the end of primary school in the Canton of Aargau, Switzerland. It tried to describe the learners’ EFL writing competence from two different perspectives (see Figure 5.1). Figure 5.1 Overview of the research questions of part I of the study The research questions I.1 and I.2 addressed the learners’ EFL writing compe‐ tence in a quantitative way by investigating what language levels the pupils reached in EFL writing, and by examining differences between the groups of learners about to enter the different educational tracks at lower secondary school. This quantitative analysis was complemented with a detailed description of the characteristics and qualities of the young EFL learners’ texts, differenti‐ ated by language level (see Figure 5.1, RQ I.3-I.5). This chapter summarises the most important findings and discusses their implications for teaching and assessing EFL writing. In addition, it highlights aspects relevant for policy makers and research. 5.1.1 The learners’ CEFR language level in EFL writing in grade six The findings presented in chapter 4.1.1 show that about 90 % of the learners reached or exceeded level A1.2 in EFL writing, which is the minimum require‐ ment for EFL writing as defined by the Swiss national standards for foreign language learning ( EDK , 2011) and the cantonal curriculum ( BKS , 2018). About 10 % of the learners did not yet achieve it. Most learners were at levels A1.2 (37 %) or A2.1 (46 %), and about 7 % of the pupils reached level A2.2 or higher. Compared to the findings by Bayer and Moser (2016), who measured the learners’ EFL writing competence at the beginning of grade six in the same canton and did not include children with individual learning objectives in the analysis, the results of this study show a slightly higher level of competence (see Figure 5.2). Figure 5.2 Comparison of the findings of this study with the findings by Bayer and Moser (2016) While the percentage of learners at level A2.2 is similar in both studies (6 %), some differences at the lower levels can be observed. In the study by Bayer 224 5 Discussion and Moser (2016), 18 % of the learners did not yet reach level A1.2, 45 % reached level A1.2 and 30 % level A2.1 (Bayer & Moser, 2016, p. 28). The reasons for these differences cannot be fully explained. Participation in both studies was voluntary (in the study by Bayer and Moser listening and reading were compulsory but writing was optional, see Bayer & Moser, 2016, p. 16). The time of data collection, however, was 4-6 months later in the study presented here, which might explain some of the differences. Furthermore, Bayer and Moser (2016) report that they conducted a standard setting for listening and reading, but not for speaking and writing, and thus conclude that the findings for these two competence areas may not be fully generalisable. However, despite the differences, the results of the two studies appear to confirm that the majority of learners in grade 6 in the Canton of Aargau reaches level A1.2 or A2.1 in EFL writing. These findings seem to be of particular interest since the preparatory work for the Swiss national standards for foreign language learning took place at a time when foreign language learning at primary school had not yet been introduced in many cantons, or only shortly before (see chapter 1.3). The experts could only predict (based on measurements at secondary school) what the pupils who would start learning English at primary school might be able to achieve (Konsortium HarmoS Fremdsprachen, 2009, p. 91). Generally, the results of this study seem to confirm that the curricular aims for EFL writing at primary school were set at a level achievable for most learners. While level A1.2 is defined as the minimum requirement, the current curriculum of the Canton of Aargau also explicitly states that the pupils are supposed to work on the next higher levels when they have accomplished the basic level ( BKS , 2018, p. 20). Similarly, the previous curriculum specified level A1.2 as the basic requirement and level A2.1 as an extended requirement for stronger learners ( BKS , 2008). The decision to keep the minimum requirement for EFL writing at level A1.2 after the transition phase, when the new school subject would be established (see chapter 1.3), appears to accord with the EFL writing competence the learners currently display. Moreover, the analysis presented in chapter 4.1.2 shows that the majority of learners about to enter the Kleinklasse or Realschule (lowest educational tracks at secondary school) is also able to reach this level. Level A2.1, on the other hand, was only reached by very few learners of this group. Therefore, it seems to be reasonable to keep level A1.2 as the minimum requirement for EFL writing at primary school, but to challenge stronger learners to try and reach level A2.1. 225 5.1 Part I: The young EFL learners’ writing competence 5.1.2 Heterogeneity in the learners’ EFL writing competence and the transition from primary to secondary school Another aspect that seems to be of particular relevance is that statistically significant differences in terms of EFL writing competence were found between the groups of learners about to enter the different educational tracks at lower secondary school (see chapter 4.1.2). The majority of learners entering a Kleinklasse or Realschule reached level A1.2 in EFL writing, the majority of learners entering the Sekundarschule reached levels A1.2 to A2.1, and the majority of learners entering the Bezirksschule reached level A2.1. This may give the secondary school teachers some guidance on what they can expect from the learners when they move from primary to secondary school. Of particular relevance in this respect may be chapter 4.1.3, which gives a detailed account of what young EFL learners’ texts at different language levels might look like, and chapter 4.1.4, which highlights the great heterogeneity that can be observed within the learners’ texts. The analysis in chapter 4.1.2, however, also shows that the EFL writing competence of learners about to enter the Kleinklasse and Realschule ranges from below A1.1 to A2.2, and that of learners about to enter the Sekundarand Bezirksschule ranges from A1.1 to above A2.2. Therefore, while there are overall significant differences in the learners’ average writing competence between the different educational tracks at secondary school, the teachers may still have to expect the learners in their classes to have largely diverging competences in EFL writing. It thus appears that the primary and the secondary school teachers are confronted with largely diverging EFL writing competences in their classes. This might pose significant challenges for the teachers when developing tasks and learning opportunities to promote the learners’ EFL writing competence. As discussed in chapters 2.4 and 2.5, lowand high-ach‐ ieving pupils seem to approach writing tasks differently and appear to have different needs with regard to instructional scaffolding. Thus, the development of learning opportunities for such a diversity of learners appears to require a considerable amount of knowledge about different options of task design as well as knowledge about the learners’ specific pre-requisites. Building up such knowledge and finding ways to apply it in classroom practice appears to be an undertaking that cannot be completed within a few days. Rather, it seems to require an on-going process of input, reflection and application. Since both primary and secondary school teachers face such challenges, and since there may often be only one English teacher at a school, without any opportunity for collaboration with other English teachers (in particular in small schools), it 226 5 Discussion might be desirable to initiate collaboration across schools. This would give the teachers the opportunity to meet, exchange experiences and ideas for teaching or to jointly attend professional development courses. Such a collaboration could enable them to address this challenge together and benefit from each other’s experience. 5.1.3 Characteristics of young EFL learners’ texts at different CEFR language levels A further key outcome of this research project is the illustration and description of the characteristics and qualities of young EFL learners’ texts at different language levels. The profiles presented in chapter 4.1.3 illustrate the large heterogeneity in the learners’ EFL writing competence. While even learners at level A1.1 (see Profile 2) seem to be able to convey some basic information in the form of a very short text, they also appear to be considerably constrained in expressing their ideas by their limited language resources. Nevertheless, the texts also reveal different strategies the learners use to overcome this difficulty, such as the use of L1 vocabulary and structures, English substitutes, or the copying of phrases from the task sheet. At level A1.2 (see Profile 3), the pupils seem to succeed better in passing on relevant information in a simple way. While the language they use is still simple and contains many errors, there are also some signs of communication with the reader, e. g. they may create some tension in a simple way or address the reader directly. At level A2.1 (see Profile 4), the pupils seem to be able to tell a simple story and provide the reader with relevant personal details. Both in terms of vocabulary and language structures, they use a mixture of simple and more complex elements. While some simple elements may be used correctly, there are still basic errors and the different grammatical structures do not yet seem to be stable. At level A2.2 (see Profile 5), the pupils appear to succeed in telling a simple and coherent story and in conveying detailed information about themselves. They use more complex and varied vocabulary, sentences, grammatical elements and cohesive devices than learners at lower levels. Although the language is increasingly used correctly, there are still basic errors in the text. Being aware of what the different language levels mean in terms of text quality appears to be relevant for primary and secondary school EFL teachers for several reasons. Firstly, a clear understanding of the kind of texts young EFL learners are able to produce may help the teachers to develop tasks and learning opportunities that match the learners’ language levels, give them the opportunity to be 227 5.1 Part I: The young EFL learners’ writing competence successful and help them develop and maintain a positive attitude towards writing. Considerations such as those presented in chapter 3.2.2 about task design may additionally support the teachers in this respect. Secondly, such descriptions may help the teachers to develop the necessary diagnostic competence to give their learners suitable feedback on how to improve their texts. As discussed in chapter 2.9.1, good feedback should indicate how well the learner has met the writing goals and how the text could be improved (Parr & Timperley, 2010). Moreover it should be specific, be based on evidence in the text and refer to depth as well as surface characteristics of text quality (Parr & Timperley, 2010). Thus, a clear understanding of the different text qualities may support the teachers when providing feedback to their learners. Thirdly, such descriptions and examples may give the teachers some guidance on what criteria they could use when assessing young EFL learners’ texts. Rating scales that are linked to the CEFR language levels, such as those presented in chapter 3.2.1, may provide further support. However, it does not seem to be possible to directly adopt such rating scales in teaching practice because they are too comprehensive. The teachers may have to select the most relevant criteria or dimensions of text quality and create assessment sheets that match the particular tasks they are using. While such an illustration of the characteristics and qualities of young EFL learners’ texts at different language levels appears to contribute to everyday teaching practice in various ways, it must also be emphasised that the descrip‐ tions of Profiles 1 to 6 presented in in chapter 4.1.3 are based on texts that are largely homogeneous in their quality. It therefore seems crucial to remind the reader why they were selected, namely because their homogeneity allowed the different language levels to be clearly illustrated. Further analyses showed that such a homogeneity is not necessarily typical of young EFL learners’ texts. On the contrary, their quality varies considerably not only as between different tasks, but also within the texts with regard to different dimensions of text quality. The following two chapters briefly summarise these findings (see chapter 4.1.4) and discuss their implications for classroom practice and research. 5.1.4 Heterogeneity within the learners’ texts: implications for teaching and research The analyses presented in the first part of chapter 4.1.4 show that considerable heterogeneity could be observed not only between different text types, but also within the learners’ texts regarding different dimensions of text quality. Two additional profiles with a within-text heterogeneity above average were 228 5 Discussion presented in order to give a more comprehensive picture of the learners’ EFL writing competence (see Profile 7 and Profile 8). The two examples showed that heterogeneity within texts may take different forms and that it may sometimes be difficult to discern this when reading a text for the first time. While one of the profiles displayed a high level of accuracy and syntactical and grammatical complexity, but a comparably low level of detail, coherence and punctuation, the other profile displayed higher communicative qualities, but less quality in terms of vocabulary, orthography and syntactical and grammatical correctness. These observations appear to have important implications for assessing young EFL learners’ texts. It seems crucial that both teachers and researchers be aware of this heterogeneity when planning the assessment or rating of the learners’ texts. As was observed during the first standard setting (see chapter 3.5.2), a quick holistic rating might lead to focusing predominantly on salient and surface features of text quality and thus result in a biased estimation of writing competence. A more reliable approach seems to be the use of semi-holistic rating scales, such as those presented in chapter 3.2.1, that get the assessors to consciously focus on different types of text quality. In teaching practice, however, as already mentioned, it does not seem to be feasible to use such a comprehensive assessment grid, and it therefore appears important for the teachers to select aspects that are relevant with regard to the writing task and the writing goals. It may also be possible to use a very simple assessment grid that lists relevant dimensions of text quality and defines how many marks the learners can get for each dimension. In a cases of uncertainty, the teachers could still consult rating grids such as those in chapter 3.2.1 or text samples as in chapter 4.1.3 and 4.1.4 as reference documents. Whatever option the teachers may select, it appears important to emphasise that assessing young EFL learners’ texts is not a simple task, since many of the texts can display a considerable within-text heterogeneity that may not be easy to identify. Besides implications for assessing EFL writing, the within-text heterogeneity of young EFL learners’ texts also seems to have implications for teaching. As discussed in chapter 5.1.3, knowing about different types of text qualities may be highly useful for giving the learners feedback and showing them how they could improve their texts. Some learners may be good at entertaining the reader or giving vivid descriptions, while others might be better in terms of linguistic text qualities. An awareness of text heterogeneity and the different pragmatic, sociolinguistic and linguistic text qualities may thus be of great use to the teachers, in particular when giving feedback to the learners. 229 5.1 Part I: The young EFL learners’ writing competence 5.1.5 Heterogeneity in the learners’ performance between different text types: implications for teaching and research The analyses presented in the second part of chapter 4.1.4 show that many learners perform differently in different tasks. At levels A1.2 and A2.1, about 43 % of the learners displayed a medium, high or very high difference in their performances of the two tasks. About half of the learners at the two language levels performed better in the e-mail task, while the other half did better in the story task. At the lowest and highest levels, a clear supremacy of one of the tasks over the other could be observed. While the learners at levels A1.1 and below performed significantly better in the e-mail task than in the story task, the learners at levels A2.2 and above did significantly better in the story task than in the e-mail task. These findings seem to indicate that the types of tasks used for gauging the learners’ EFL writing competence are of central importance. While the e-mail used in this study appears to have been a suitable task up to level A2.1, it may have restricted the high-level learners in displaying their ability, even though they were encouraged to describe certain aspects in detail and write about further topics of their choice (see chapter 3.2.2). The story, on the other hand, seems to have stretched the writing competence of the A1.1-level learners to the limit, while at the same time providing high-level learners with the opportunity to show their best. Therefore, it appears crucial that careful consideration be given to the selection, development and wording of writing tasks, both in classroom practice and research. In particular when using a multi-level approach to gauging the learners’ writing competence (see chapter 3.2.2), caution should be exercised to ensure that the writing tasks adequately cover all language levels. Furthermore, since a considerable number of learners performs differently on different tasks, it seems advisable to use a combination of different tasks and genres when assessing the learners’ EFL writing competence, rather than to use only one single task (see also chapter 2.3.2). This may help to avoid biased results (Bouwer et al., 2014; Furger & Lindauer, 2013), and also cater for the learners’ different preferences. 5.1.6 Communicative effect: a central dimension of EFL writing The last two aspects of the young EFL learners’ writing competence that were addressed in this first part of the study were the communicative effect and coherence in young EFL learners’ narrative texts. As discussed in chapter 2.8.1, both aspects do not seem to have been well researched in a young EFL learner 230 5 Discussion context up to now. Also, they do not seem to be considered very often in teaching practice (see findings in chapter 4.2.1, RQ II .4). A statistical analysis showed that all language levels except for the combina‐ tion A2.1-A2.2 and above were statistically significantly different from each other in terms of the scores for communicative effect / creativity (see chapter 4.1.5). A small qualitative analysis with five texts from each language level was therefore conducted in order to investigate whether and in what way young EFL learners at different language levels create a communicative effect in their narrative texts. The analysis showed that even some of the texts at the lowest level (below A1.1) contained one or two elements that created a very small communicative effect. It was mainly direct speech that caught the reader’s attention, such as kom on däd (= Come on, Dad! ), or once the creation of tension. At levels A1.1 and A1.2, the number of elements that created a communicative effect increased to one to seven elements. While at the lowest level a story ending was missing in all five texts, the learners at these levels almost all invented their own story ending. Additionally, they used linguistic and textual elements as well as tension, unexpected turns and emotions to create a communicative effect. At levels A2.1 and A2.2, the number of elements further increased to up to 22 elements in a text. Similar means were used to those employed on levels A1.1 and A1.2, but clearly more frequently. Some of the texts additionally contained an amusing element or some interaction with the reader. This small analysis shows that even learners at levels A1 and A2 are able to create a communicative effect in narrative texts, and that they use different means to do so, some to a smaller, and others to a larger extent. Considering the discussion in chapter 1.2 about the relevance of a communicative and functional use of the foreign language, such findings seem to be of paramount importance. If writing is regarded as an activity that should lead to the creation of a written product which fulfils a particular function (see construct definition in chapter 2.1), it seems central that not only the teachers should be aware of this aspect, but that the learners are also made aware of it, for example through storytelling and giving them a vivid example of captivating elements. By simple means, such as having the learners repeat key phrases while telling a story, or simple role-plays, the learners can become familiar with useful language they can use themselves when writing their own stories. A few simple expressions for unexpected turns (e. g. suddenly), emphasis (e. g. The lion was very, very big! ), direct speech (e. g. Hold on! ) or emotions (No way! ) may already give the learners a small repertoire of elements they can use. 231 5.1 Part I: The young EFL learners’ writing competence In addition to familiarising the learners with useful language, brief sequences of brainstorming ideas and planning the text before they start writing may also support the learners in the process of reflecting on what purpose the text should serve, and how this could be achieved (see chapters 2.4.1 and 2.4.3). In particular less able writers seem to appreciate such a preparation (Griva et al., 2009). As research by Donovan and Smolkin (2002) has shown, even elementary school children were able to consider the purpose and the intended readers when composing texts (see chapter 2.3.1). 5.1.7 The importance of coherence in young EFL learners’ texts The last analysis concerned the coherence in young EFL learners’ narrative texts. Statistically significant differences between the language levels were also found for these scores. While at lower levels the learners’ texts only contained few elements of a simple narrative structure, higher-level texts contained most or all elements (see chapter 4.1.6). Moreover, the number of gaps in the texts decreased with higher language levels. At the lowest language levels, the gaps tended to negatively affect the comprehensibility and required the reader to read the text slowly and carefully in order to understand the course of the story. At levels A1.2 and A2.1, the gaps did not affect comprehensibility, but sometimes they required the reader to mentally compensate for missing elements. At level A2.2, the texts were largely coherent, with only minor gaps. As in the case of the communicative effect, coherence seems to be an aspect of text quality that is not frequently addressed in a young EFL learner classroom (see findings in chapter 4.2.1, RQ II .4). Discussing how to structure a text, however, and what elements to include, may help the learners to avoid problems with the organisation of a text (Griva et al., 2009, see chapter 2.4.3), and improve its comprehensibility. With regard to different genres, the teacher could use a model text to make the learners aware of its genre-specific structure and its key elements (see chapters 2.3.2 and 2.9.1). Additionally, a brief planning phase, in which the learners make a list or mindmap of the elements they want to include, and decide in what order they should appear, may support the learners. Less able writers who, according to Griva et al. (2009), do less pre-planning than stronger learners and may not always succeed in doing so, may need specific support in this respect (see chapter 2.4.3). Furthermore, rereading may play an important role with regard to the improvement of text coherence, in particular when a text is read out to a peer who may afterwards give feedback on whether the text is comprehensible, or whether there are any gaps that should be filled. As discussed in chapter 2.4.6, 232 5 Discussion less able learners in particular seem to need specific support in order to notice inconsistencies in terms of content and coherence (McCutchen et al., 1997). According to research by Beal et al. (1990), the ability of young L1 learners to locate and revise inconsistencies on text level can improve if they are trained to ask specific questions about the content when reading and revising their texts. Therefore, a procedure like the one described above may help the learners develop an awareness of what is important for the reader and support them in revising their texts. 5.2 Part II: Current teaching practices and the learners’ perception of EFL writing Part II of this study investigated how EFL writing is currently taught in the primary schools in the Canton of Aargau and how it is perceived by the learners (Figure 5.3). It was an explorative study that aimed at serving as a starting point for discussing different methodological options for teaching EFL writing to young learners, and as a basis for further research in this field. 233 5.2 Part II: Current teaching practices and the learners’ perception of EFL writing Figure 5.3 Overview of the research questions of part II of the study This chapter discusses a selection of findings from chapter 4.2 that appear to be of particular relevance to this purpose. 5.2.1 The role of EFL writing as a communicative competence When asked about the importance they assign to writing compared to other teaching components, the teachers indicated that they regarded writing as less important than the oral language skills listening and speaking, and also as less important than reading (see chapter 4.2.1, RQ II .1). This finding seems to correspond with the slightly lower curricular expectations for writing than for the other language skills, as defined in the national standards for foreign language learning ( EDK , 2011) and the cantonal curriculum ( BKS , 2018). The fact, however, that the language resource spelling is, on average, consid‐ ered as slightly more important than writing as a skill appears to contradict the curriculum, which expects the language resources to be subservient to the communicative competences (D- EDK , 2016, p. 11). This, and also the high variability in the teachers’ responses with regard to spelling, might reflect the teachers’ uncertainty about the role of EFL writing at primary school which led to the initiation of this project (see Introduction). Therefore, it seems important for the teachers to be provided with clear guidance on the role and nature of EFL writing at primary school. It is hoped that this dissertation may serve as 234 5 Discussion one of the means to this end. Additionally, teacher training and professional development courses may be able to make valuable contributions in this respect. 5.2.2 The role of learning strategies in classroom practice Another finding, which is not specifically related to EFL writing, but neverthe‐ less appears to be of great importance, is that the teachers, on average, consider learning strategies as the least important of the different teaching components (see chapter 4.2.1, RQ II .1). This appears remarkable, considering that strategy instruction is a well-re‐ searched teaching component that has repeatedly been shown to be one of the most effective means of improving the learners’ performance. A meta-analysis by Plonsky (2011) on second and foreign language strategy instruction, for example, showed that it is highly effective, in particular at a young age, up to 12 years ( ES = 1.29), and for developing skills and resources such as speaking ( ES = 0.97), reading ( ES = 0.74), pronunciation ( ES = 0.70), vocabulary ( ES = 0.64) and writing ( ES = 0.42). Similarly, as discussed in chapter 2.9.1, Graham et al. (2012) and Koster et al. (2015) found a significant positive effect of strategy instruction on young L1 learners’ writing skills ( ES = 1.02 and ES = 0.96, respectively). As discussed in chapter 2.5.2, less able writers seem to use strategies less frequently and less effectively than their more capable peers (Boscolo, 2008; Griva et al., 2009) and may therefore need specific support with regard to the kind of strategies they could use and with how to go about it. It therefore appears to be of paramount importance for strategy instruction to be promoted as a key element of foreign language teaching methodology, and for the learners to be specifically instructed on how to use such strategies for learning a language. 5.2.3 The role of pragmatic text qualities in classroom practice Another finding, which was already briefly mentioned in chapters 5.1.6 and 5.1.7, is the role of pragmatics in classroom instruction. As described in chapter 4.2.1 ( RQ II .4), pragmatic text qualities such as the communicative effect (e. g. how to write a funny, sad or captivating story), coherence (how to structure a text) or level of detail (how to describe something in detail) are only rarely discussed or addressed in a young EFL learner classroom. In the assessment of young EFL learners’ texts, however, pragmatic text qualities appear to be considered more frequently. 12 of the 19 teachers, for example, indicated that they would assess the content of a text, and seven teachers reported that they 235 5.2 Part II: Current teaching practices and the learners’ perception of EFL writing would assess coherence (see chapter 4.2.1, RQ II .8). It was also found that more than two thirds of the teachers used a combination of different assessment criteria from the macro, meso and micro level of text quality when assessing the young EFL learners’ texts (see chapter 4.2.1, RQ II .8). These findings seem to indicate that while there is an awareness of different types of text quality when assessing young EFL learners’ texts, classroom instruction focuses more on linguistic aspects. It may therefore be advisable to also include pragmatic aspects in classroom instruction (see chapters 5.1.6 and 5.1.7 for some suggestions regarding the communicative effect and coherence). 5.2.4 The role of different genres when teaching EFL writing The analysis in chapter 4.2.1 about different tasks and genres used in class ( RQ II .5) shows that the teachers use a wide variety of tasks and genres when teaching EFL writing. The most frequently applied genre was factual texts, followed by narratives and correspondence. A few teachers also had the learners write an argumentative text, or a dialogic text such as an interview or roleplay. These findings seem to correspond with the curriculum, which mainly expects the primary school EFL learners to work with factual and narrative texts and correspondence ( BKS , 2018). Argumentative texts are only mentioned in the curriculum from level B1 onwards ( BKS , 2018), which might explain why this genre is only rarely dealt with when teaching EFL writing at primary school. Hallet (2016), however, argues that even primary school children could try to write very simple argumentative texts (see chapter 2.3.2). It might thus be an interesting task for teachers to find out whether and how this could be implemented in class. As described in chapter 4.2.1 ( RQ II .2), learning about different genres is not very prominent among the aims primary school teachers pursue when teaching EFL writing. They sometimes address genre-specific aspects in class (see 4.2.1, RQ II .4), study a similar text with the class before the learners start writing or offer a text model while writing (see 4.2.1, RQ II .5). This seems to correspond with the different phases of generic learning as discussed by Hallet (2016), which move from a mainly intuitive encounter with different genres in the first years of foreign language learning to a more explicit use of genre-specific terms and a more conscious categorisation of genres later on (see chapter 2.5.2). The learners’ experience with the different genres in the first years of foreign language learning could thus serve as a starting point for a more explicit use of genres towards the end of primary school and at lower secondary school. 236 5 Discussion 5.2.5 Elements of the process and genre approach The findings presented in chapters 4.2.1 and 4.2.2 also provide information on the extent to which the various elements of the process and genre approach are applied in teaching practice. Commonly used elements of the process approach include addressing the topic the children will write about in class, compiling words and sentences before starting to write, different types of scaffolding (e. g. looking up words or asking the teacher or peers for help) and feedback on the written product. Less frequently applied are elements such as collecting ideas for writing, discussing how to structure a text and publishing the texts to a real audience. With regard to the genre approach, the findings show that there are some elements that are applied, but they do not seem to be as frequently used as certain elements of the process approach (see chapter 4.2.1, RQ II .7). About half of the teachers reported that their learners can use a text model while writing, and one third or fewer get the learners to study a sample text before they start writing, or consider genre-specific aspects of a texts (see Table 4.13). In summary, these findings show that many elements of the process approach and some elements of the genre approach are applied in classroom practice. It thus seems that the teachers are well aware of different ways of supporting young learners in the process of developing their EFL writing competence, in particular with regard to the writing process. The concept of genre-based teaching might be an approach that could complement the various teaching procedures that are already in use. 5.2.6 Motivation, resources and task demands: three important elements affecting the learners’ perception of EFL writing Part II of the study also investigated the learners’ perception of EFL writing. They were, for example, asked what they like and dislike about writing in English, or what topics they would like to write about, and why. The learners’ an‐ swers to these questions revealed that motivation, resources and task demands seem to have a considerable influence on how EFL writing is perceived by the learners. First, the learners reported that motivation in various forms contributes to their perception of EFL writing (see chapter 4.2.1, RQ II .11 and RQ II .12). In the interviews, the learners mentioned that having something interesting to tell (e. g. I had such a funny idea), creativity (e. g. you can always invent new things), text design (e. g. complementing a text with drawings or decoration), writing to 237 5.2 Part II: Current teaching practices and the learners’ perception of EFL writing a friend, interest in the topic, or the intention to improve an existing product had the effect that they perceived EFL writing as something positive. Secondly, the learners mentioned that the availability of resources affected their perception of EFL writing. In particular, they reported that having or not having enough ideas and language resources has an impact on whether they like or dislike writing in English. Learners who reported that they had had good ideas spoke positively about the writing tasks, whereas not having enough ideas was perceived as stressful. Similarly, the learners perceived writing positively if they had enough language resources to complete the task, while learners who reported that they lacked the necessary resources did not like writing in English. A third aspect that was mentioned was the task demands. Some learners reported that the expected length of a text or a time limit affected their perception of EFL writing. One girl, for example, feared that she would not be able to come up with the necessary ideas when having to work under time pressure (see chapter 4.2.1, RQ II .11). Moreover, while some learners preferred open tasks that allowed them to express their own ideas, this openness was perceived as difficult by other learners who preferred more guidance. These findings seem to correspond with the research by Nolen (2007), who found that L1 elementary school learners mentioned mastery, interest, enjoyment, creativity, choice and self-expression as the reasons for their motivation to write (see chapter 2.7.1). In contrast to L1 writing, however, the language resources appear to play a more prominent role in EFL writing. This was also observed by Griva et al. (2009) and Griva and Chostelidou (2013), who found that the young EFL learners’ limited language resources were a major challenge for them, and that less able learners were more anxious about writing in the foreign language than their more capable peers (chapters 2.4.4 and 2.5.1). Moreover, Griva et al. (2009) found that less able learners showed greater difficulty in generating ideas (see chapter 2.4.1). Therefore, if classroom instruction aims at developing and maintaining a positive attitude towards EFL writing, it seems to be of paramount importance for the young EFL learners’ specific prerequisites to be considered. As discussed in chapter 2.3.3, this may include the development of meaningful, communicative tasks that cater for different language levels in that they give strong learners the opportunity to be creative and express their ideas, while also providing sufficient support for less able learners. In particular, support with regard to language resources and the generation of ideas, as well as achievable task demands, appear to be aspects that are greatly appreciated by the learners (see also chapters 2.4.1 and 2.4.4). While many teachers seem to support their learners with regard to language resources (see chapter 5.2.5), specific activities to support the generation of ideas are less frequent. Thus, a more 238 5 Discussion frequent use of such activities may have the potential to increase the learners’ confidence and contribute to a positive perception of EFL writing at primary school. 5.3 Part III: Predictors of the learners’ EFL writing competence Part III of this study investigated the extent to which different individual and educational factors are predictors of the learners’ EFL writing competence (see Figure 5.4). This chapter summarises the findings presented in chapter 4.3, compares them to findings from other research studies and discusses their implications for classroom practice. Figure 5.4 Overview of the research questions of part III of the study 5.3.1 The learners’ self-efficacy As described in chapter 4.3.1 ( RQ III .1), both the learners’ self-efficacy in writing in English and in learning English were found to be statistically significant predictors of the learners’ EFL writing competence. A multiple regression model that encompassed the two variables, F(2, 317) = 77.30, p < .001, accounted for 32.4 % of the variation in EFL writing competence (adjusted R 2 ), which is a large effect size (f = 0.69) according to Cohen (1988). Both variables added statistically 239 5.3 Part III: Predictors of the learners’ EFL writing competence significantly to the regression model, with β = 0.40 (p < .001) for the learners’ self-efficacy in writing in English and β = 0.22 (p < .001) for the learners’ general self-efficacy in learning English. Both variables must therefore be considered strong predictors of the young EFL learners’ writing competence. Learners who have confidence in their ability to successfully complete a writing task in English, and learners who have confidence in their ability to learn English, display a higher EFL writing competence than learners who are less confident in this respect. These findings show that the learners’ self-efficacy has a significant effect on young learners’ writing competence, not only in an L1 context (Bulut, 2017; Pajares & Valiante, 1997; Troia et al., 2013, see chapter 2.7.1), but also in an EFL context. Further‐ more, these findings seem to confirm what has already been discussed in the preceding chapter, namely the importance of providing young EFL learners with opportunities to develop a sense of mastery in EFL writing. Such a ‘sense of competence’ (Boscolo, 2009, p. 302) may thus not only increase the learners’ motivation, but also have a positive effect on their writing development. 5.3.2 The learners’ extra-curricular use of English Like the learners’ self-efficacy, the extra-curricular use of English was found to have a significant effect on the learners’ EFL writing competence (see chapter 4.3.1, RQ III .2). A regression analysis showed that it was a statistically significant predictor, with β = 0.44 (p < .001) and an overall model fit of F(1, 318) = 74.07, p < .001. It accounted for 18.6 % of the variation in EFL writing competence, which is a large effect size (f = 0.48) according to Cohen (1988). This finding confirms that the extra-curricular use of English affects EFL writing not only at secondary school (Olsson, 2011, see chapter 2.9.2), but also at primary school, and corresponds with similar findings with regard to its effect on vocabulary, listening and reading skills (Sylvén & Sundqvist, 2012). Even though the learners’ extra-curricular use of English cannot be directly influenced by the teachers, the finding nevertheless confirms an observation many teachers seem to make, and may encourage them to suggest different extra-curricular activities to their learners in order to help them improve their language competence. 5.3.3 Different educational factors In order to examine the effects of different educational factors on the learners’ EFL writing competence, a multi-level analysis was conducted with pupils 240 5 Discussion nested within classes (see chapter 4.3.2). Because the learners’ responses to questions about classroom instruction were not reliable enough (see chapters 3.4.4 and 3.6.4), only the teachers’ responses were used for the analysis, which considerably limited the options and the power of the analysis (see chapter 4.3.2). The null model showed that 12.7 % of the variance of the learners’ EFL writing competence were due to differences on the class level. These 12.7 % were examined in more detail, and a random intercepts model with one level 2 predictor showed that 4.7 percentage points of this class level variance could be traced back to differences in the proportion of highand low-achieving pupils in the classes. Further random intercepts models with two level 2 predictors were conducted to include different educational factors, but none of these factors could be shown to significantly contribute to the explanation of the class level variance. The influence of the frequency of text composition in class, however, was close to being significant, and may therefore be an educational factor worth further investigation. While studies with an experimental design showed that teaching components such as pre-writing activities and feedback have an effect on the learners’ L1 writing competence (Graham et al., 2012, see chapter 2.9.1), a questionnaire study in an EFL context with learners at secondary school (grades 8 to 10) found that the pre-writing activities and the number of texts written in the foreign language had a significant effect on the learners’ EFL writing competence (Porsch, 2010). Therefore, further research with either an experimental design or a large enough sample size at class level appears to be necessary in order to investigate whether these findings can also be confirmed in a young EFL learner context. 241 5.3 Part III: Predictors of the learners’ EFL writing competence Conclusion As mentioned in the introduction, this project is a subject-specific educational research study that aimed at providing EFL teachers with empirical evidence to support them in their task of promoting young EFL learners’ writing competence in an effective and age-appropriate manner. As a first step, a comprehensive model of writing competence for young EFL learners was developed which provided a framework for presenting the current state of research on EFL writing in a young learner context (see chapter 2). The study itself focussed on three different aspects: the young EFL learners’ writing competence (part I), current teaching practices and the learners’ perception of EFL writing (part II ), and factors which are predictors of the learners’ EFL writing competence (part III ). The young EFL learners’ writing competence Part I of the study gave important insights into the young EFL learners’ writing competence, which seem to be relevant, not only for the teachers, but also for teacher education, research and educational ministries. The study showed that at the time of data collection (4-6 months before the end of the school year) about 90 % of the learners reached or exceeded the minimum curricular requirement for EFL writing (i. e. level A1.2), and 10 % did not yet reach it (see chapter 4.1.1). Included in these figures are learners with individual learning objectives, who do not have to reach the official curricular aims, and learners with learning difficulties such as dyslexia. If these pupils are excluded from the analysis, the percentage of learners not reaching level A1.2 decreases to 7.6 %. In summary, 10 % of the learners were at level A1.1 or below, 37 % at level A1.2, 46 % at level A2.1 and 7 % at level A2.2 and above. Statistically significant differences in terms of EFL writing competence were found between the groups of learners about to enter the different educational tracks at secondary school (see chapter 4.1.2). The majority of learners about to enter the Kleinklasse or Realschule (lowest educational tracks at secondary school) was at level A1.2, the majority of learners about to enter the Sekundarschule (medium track) was at levels A1.2 to A2.1, and most learners about to enter the Bezirksschule (highest track) were at level A2.1. It could thus be concluded that the great majority of learners achieves the minimum requirement set by the curriculum, and that this minimum requirement is also achievable for most learners who are about to enter the lowest educational tracks at secondary school. Stronger learners generally reach higher language levels in EFL writing, which seems to correspond with the curriculum, which expects the learners to work on the next higher levels once they have achieved the basic level ( BKS , 2018). These quantitative findings were complemented with examples and descrip‐ tions of the young EFL learners’ texts in order to illustrate what the different language levels mean in terms of text quality. First, six profiles (one from each language level) were presented which were largely homogeneous in their ratings of the different text qualities (see chapter 4.1.3). They were selected because their homogeneity allowed a clear illustration of the different language levels, which seemed to be particularly important for supporting the teachers in developing a clear understanding of the differences in terms of text quality between the language levels. Since an analysis of within-text heterogeneity, however, had shown that such homogeneity does not adequately represent what is to be expected of young EFL learners in general, two further profiles with a higher within-text heterogeneity were presented in order to provide a more comprehensive picture of the learners’ EFL writing competence (see chapter 4.1.4, part 1). These profiles illustrated that within-text heterogeneity can take different forms and that it is sometimes difficult to identify. While the first profile showed a high level of grammatical accuracy and complexity, and a rather low level of detail, coherence and punctuation, the second profile showed high communicative qualities, but less quality with regard to vocabulary, or‐ thography and grammatical correctness. It was thus emphasised that within-text heterogeneity should be taken into account when assessing young EFL learners’ texts, in particular since the standard setting (see chapter 3.5) had also shown that a quick holistic rating may lead to focussing mainly on salient and surface features of text quality, and thus result in a biased estimation of the learners’ EFL writing competence. The use of a selection of different criteria that focus on different dimensions of text quality, or the use of semi-holistic scales such as those presented in chapter 3.2.1, were therefore recommended, rather than holistic ratings. In addition, since a considerable number of learners performed differently on the two tasks (see chapter 4.1.4, part 2), it seems advisable to use a combination of different tasks and genres in order to draw conclusions about the learners’ EFL writing competence. In addition to these descriptions of key characteristics of young EFL learners’ texts, two more specific analyses were conducted that focused on the commu‐ nicative effect and the coherence in young EFL learners’ narrative texts. Both aspects do not seem to have been well researched in a young EFL learner context up to now (see chapter 2.8.1) and have been shown to be only rarely addressed in classroom practice (see findings presented in chapter 4.2.1, RQ II .4). In terms 244 Conclusion of the communicative effect in young EFL learners’ narrative texts, it was found that even texts at level A1.1 contained a few elements that created a small communicative effect (see chapter 4.1.5). The number of elements generally increased with language level, to more than 20 elements in some of the texts at level A2.2. However, a large heterogeneity could also be observed here. The ratings for communicative effect / creativity (measured on a scale from 0 to 4, see chapter 3.2.1) of the texts at level A2.1, for example, ranged from 0 to 4, and those of the texts at level A2.2 from 1 to 4. Table 4.2 gives an overview and some examples of the kind of elements the learners used to create a communicative effect in their narrative texts. It was argued that if writing is regarded as a activity that leads to the creation of a product that has a particular function (see construct definition in chapter 2.1), the learners should be made aware of this aspect, be familiarised with language that can be used for this purpose and supported accordingly when planning their texts. Some simple ways of putting this into practice were presented in chapter 5.1.6. With regard to coherence in young EFL learners’ texts (see chapter 4.1.6), it was found that the higher the learners’ language level, the more complete the texts with regard to the different elements of a simple narrative structure, and the fewer gaps contained in the texts. At levels A1.1 and below, gaps tended to negatively influence the comprehensibility of a text and required the reader to read slowly and carefully in order to understand the course of the story. At level A1.2, the gaps did not affect comprehensibility, but the reader was still sometimes required to mentally compensate for missing elements. Different suggestions for addressing the aspect of coherence in class were presented in chapter 5.1.7. Current teaching practices Part II of the study investigated how EFL writing is currently taught in the primary schools in the Canton of Aargau, Switzerland, and what the learners’ perception of EFL writing is. It was an explorative study that aimed at serving as a starting point for discussing different methodological options for teaching EFL writing to young learners. The analysis showed that the teachers already apply many elements that can be regarded as good teaching practice. For example, they apply various elements of the process approach such as pre-writing activities (e. g. addressing the topic in class and compiling words and sentences before starting to write), scaffolding (e. g. looking up words and peer and teacher support) and feedback on the written product (see chapter 4.2.1, RQ II .6 and II .7). A few teachers also use elements of the genre approach, such as studying a sample text, discussing genre-specific aspects in class or using a text model as a template to create an 245 Conclusion own text. Less frequently applied are elements such as collecting ideas on what to write about, discussing how to structure a text, or publishing the texts to a real audience. The teachers also seem to use a wide variety of tasks and genres when teaching EFL writing (see chapter 4.2.1, RQ II .5). The most frequently used genres were factual texts, narratives and correspondence. Table 4.9 gives an overview of the many different products that were created in the participating classes in grade six. Explicitly learning about different genres, however, appears to be less frequent (see chapter 4.2.1, RQ II .2). This may be an aspect that could further support the learners in developing their EFL writing competence. Furthermore, many teachers appear to use a combination of different criteria when assessing the learners’ texts (see chapter 4.2.1, RQ II .8). The most prominently used criteria are aspects of the communicative purpose of the text (e. g. content, comprehensibility or creativity) and language mechanics (orthography). About half of the teachers mentioned that they also focus on vocabulary, grammar, syntax and coherence. As discussed in chapter 4.1.4, the use of different pragmatic, sociolinguistic and linguistic criteria appears to be central when assessing young EFL learners’ texts. Therefore, it seems important for the teachers to be aware of the different dimensions that can be assessed, so that they can purposefully select those aspects that are most relevant for the particular task at hand. While pragmatic aspects or the communicative purpose of a text seem to be frequently considered when assessing young EFL learners’ texts, the teachers also reported that they would only rarely address this aspect in class (see chapter 4.2.1, RQ II .4). Therefore, since the Swiss national standards for foreign language learning ( EDK , 2011) as well as the cantonal curriculum ( BKS , 2018) regard the ability to use the language for oral and written communication as the key aim of foreign language learning, it seems central to also include aspects such as the communicative effect (e. g. how to write a funny, sad or captivating story), coherence (e. g. how to structure a text), or the level of detail (e. g. how to describe something in detail) in teaching practice (see chapters 5.1.6 and 5.1.7 for some suggestions on how young EFL learners could be made aware of such aspects). A further key finding appears to be that strategy instruction is not very prominent in teaching practice (see chapter 4.2.1, RQ II .1). Considering that it has repeatedly been shown to be one of the most effective means of improving the learners’ language competences (see discussion in chapter 5.2.2), it appears to be of paramount importance to promote strategy instruction as a key element of EFL writing instruction, and of foreign language teaching methodology in general. 246 Conclusion Lastly, the results also showed that the primary school EFL teachers give priority to listening, speaking and reading over writing (see chapter 4.2.1, RQ II .1), which seems to accord well with the curricular requirements, which expect a slightly lower language level of the learners for writing than for the other language competences. However, there also seems to be some uncertainty regarding the role of spelling. It therefore appears important to provide the teachers with guidance on how to put into practice the curricular expectation that language resources should be subservient to the communicative compe‐ tences ( BKS , 2018). The learners’ perception of EFL writing These analyses of current teaching practices were complemented with informa‐ tion from the learner questionnaire and interviews (see chapter 4.2.2) in order to also include their view of EFL writing. Almost 90 % of the learners reported that they regarded learning to write English sentences and texts as important or quite important (see chapter 4.2.2, RQ II .9). 42 % of the learners had a high confidence in their ability to write in English, 52 % a medium and 6 % a low confidence (see chapter 4.2.2, RQ II .19). The analyses of the learner interviews showed that there are three main factors that appear to influence the learners’ perception of EFL writing, namely motivation, resources and task demands (see chapter 5.2.6). The learners reported that aspects such as having something interesting to tell, creativity and interest in the topic had the effect that they perceived EFL writing as something positive. In addition, having or not having sufficient ideas and language resources greatly affected whether they like writing in English or not. And lastly, they also reported that task demands such as the expected text length, time pressure or the amount of freedom and guidance they had in a writing task had an influence on their perception of EFL writing. It was therefore concluded that careful consideration of these aspects might play a central role in developing and maintaining a positive attitude among the learners towards EFL writing. Predictors of the learners’ EFL writing competence Part III of the study showed that both the young learners’ self-efficacy and their extra-curricular use of English are statistically significant predictors of their EFL writing competence. These findings confirm similar findings from a secondary school and L1 context (e. g. Bulut, 2017; Olsson, 2011) and show that they also apply in a young EFL learner context. The fact that the learners’ self-efficacy is a significant predictor of the their EFL writing competence seems to confirm the importance of giving the learners opportunities to be successful, at whatever 247 Conclusion language level they may be (see chapter 2.3.3), and the extra-curricular use of English might give the individual learners further opportunities to improve their language competence, including their EFL writing skills. With regard to educational factors such as pre-writing activities, scaffolding, feedback or the frequency of text composition in class, no conclusive statistical results can be presented, since the data from the learners could not be used for the analysis and the small sample size at class level limited the power of the analysis (see chapter 4.3.2). The frequency of text composition in class, however, showed results which come close to being significant, and may therefore be a factor worth further investigation. Outlook It is hoped that the findings of this research project can contribute to the development of English language teaching at primary school in various ways: First and foremost, it is hoped that the study can contribute to the professional development of many primary and secondary school EFL teachers and support them in their challenging task of promoting young EFL learners’ writing competence. Moreover, it is hoped that it can contribute to an in-depth understanding of the learners’ and teachers’ needs with regard to the learning and teaching of EFL writing and help those working in teacher education to plan suitable and effective teacher training programmes. And thirdly, it is hoped that researchers, policy makers and any other EFL specialists can also benefit from the insights and empirical evidence provided in this research report. 248 Conclusion References Arteaga-Lara, H. M. (2017). Using the process-genre approach to improve fourth-grade EFL learners' paragraph writing. Latin American Journal of Content and Language Integrated Learning, 10(2), 217-244. https: / / doi.org/ 10.5294/ laclil.2017.10.2.3 Augst, G., Disselhoff, K., Henrich, A., Pohl, T., & Völzing, P.-L. (2007). Text - Sorten - Kompetenz. Eine echte Longitudinalstudie zur Entwicklung der Textkompetenz im Grundschulalter. Frankfurt am Main: Peter Lang. Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press. Bachmann, T., & Becker-Mrotzek, M. (2017). Schreibkompetenz und Textproduktion modellieren. In M. Becker-Mrotzek, J. Grabowski, & T. Steinhoff (Eds.), Forschung‐ shandbuch empirische Schreibdidaktik (pp. 25-54). Münster: Waxmann Verlag. Bader-Lehmann, U. (2007). Teaching English in primary schools. - Herausforderungen an die Sprachdidaktik und die Lehrerbildung. Beiträge zur Lehrerbildung, 25(2), 241-254. Bader, U. (2006). Englisch an der Primarschule im Kanton Aargau. Die Situation vor Beginn des Pilotprojekts 2006 / 7 und der Ausbildung der Lehrkräfte. In H.-U. Grunder (Ed.), PrimarlehrerIn der Zukunft (pp. 88-93). Baltmannsweiler: Schneider Verlag Hohengehren. Bader, U., & Schaer, U. (2006). Evaluation Englisch in den 6. Klassen Appenzell Inner‐ rhoden 2005. Zofingen: Pädagogische Hochschule der Fachhochschule Nordwestsch‐ weiz. http: / / www.ai.ch/ dl.php/ de/ 0cqlr-ovtpgv/ Evaluationsbericht_Englisch_an_der _Primarschule_6._Klasse.pdf Bader, U., & Trüb, R. (2020). Guiding young EFL learners from comprehension to production: Theoretical background and possible applications of the pre-while-post framework [Manuscript in preparation]. Professur für Englischdidaktik und ihre Disziplinen, Institut Primarstufe, Pädagogische Hochschule FHNW. Bai, B., & Guo, W. (2019). Motivation and self-regulated strategy use: Relationships to primary school students’ English writing in Hong Kong. Language Teaching Research, 1-22. https: / / doi.org/ 10.1177/ 1362168819859921 Bayer, N., & Moser, U. (2016). Evaluation der Englischkompetenzen im Kanton Aargau. Englischkompetenzen auf der Primarstufe und auf der Sekundarstufe I. Schlussbericht zuhanden des Departements Bildung, Kultur und Sport des Kantons Aargau. Zürich: Institut für Bildungsevaluation. Assoziiertes Institut der Universität Zürich. https: / / e dudoc.ch/ record/ 130893 Beal, C. R., Garrod, A. C., & Bonitatibus, G. J. (1990). Fostering children's revision skills through training in comprehension monitoring. Journal of Educational Psychology, 82(2), 275-280. https: / / doi.org/ 10.1037/ 0022-0663.82.2.275 Becker-Mrotzek, M. (2004). Schreibentwicklung und Textproduktion. Der Erwerb der Schreibfertigkeit am Beispiel der Bedienungsanleitung. Radolfzell: Verlag für Gesprächs‐ forschung. Becker-Mrotzek, M., & Böttcher, I. (2014). Schreibkompetenz entwickeln und beurteilen (5th ed.). Berlin: Cornelsen Schulverlage. Behrens, U. (2017). Vorschule und Primarstufe. In M. Becker-Mrotzek, J. Grabowski, & T. Steinhoff (Eds.), Forschungshandbuch empirische Schreibdidaktik (pp. 75-88). Mün‐ ster: Waxmann Verlag. Bereiter, C. (1980). Development in writing. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 73-96). Hillsdale: Lawrence Erlbaum Associates. Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Mahwah: Lawrence Erlbaum Associates. Berninger, V. W., Abbott, R. D., Augsburger, A., & Garcia, N. (2009). Comparison of pen and keyboard transcription modes in children with and without learning disabilities. Learning Disability Quarterly, 32(3), 123-141. https: / / doi.org/ 10.2307/ 27740364 BKS (2008). Lehrplan Englisch für die Primarschule im Kanton Aargau. Aarau: Departe‐ ment Bildung, Kultur und Sport des Kantons Aargau. https: / / www.schulen-aargau.c h/ regelschule/ unterricht/ lehrplan-lehrmittel/ bisheriger-lehrplan/ sprachen/ englisch BKS (2014). Lehrplan für die Volksschule des Kantons Aargau. Aarau: Departement Bildung, Kultur und Sport des Kantons Aargau. BKS (2016). Lehrplan Volksschule. Primarschule und Oberstufe. Aarau: Departement Bildung, Kultur und Sport des Kantons Aargau. https: / / www.schulen-aargau.ch/ rege lschule/ unterricht/ lehrplan-lehrmittel/ bisheriger-lehrplan BKS (2018). Aargauer Lehrplan Volksschule. Gesamtausgabe. Aarau: Departement Bil‐ dung, Kultur und Sport des Kantons Aargau. https: / / ag.lehrplan.ch/ downloads.php BKS (2020). Handreichung Beurteilen in der Volksschule. Aarau: Departement Bildung, Kultur und Sport des Kantons Aargau. https: / / www.schulen-aargau.ch/ media/ schule n-aargau/ unterricht/ pruefen-beurteilen/ bksvs-hr-beurteilen-vs.pdf BKZ, NW-EDK, & EDK-Ost (2008). Lingualevel. Instrumente zur Evaluation von Fremd‐ sprachenkompetenzen. 5.-9. Schuljahr. Bern: Schulverlag plus. Bortz, J., & Döring, N. (2006). Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler (4th ed.). Berlin: Springer-Verlag. Boscolo, P. (2008). Writing in primary school. In C. Bazerman (Ed.), Handbook of research on writing. History, society, school, individual, text (pp. 359-379). New York: Lawrence Erlbaum Associates. 250 References Boscolo, P. (2009). Engaging and motivating children to write. In R. Beard, D. My‐ hill, M. Nystrand, & J. Riley (Eds.), The SAGE handbook of writing development (pp. 300-312). London: SAGE Publications. Bouwer, R., Béguin, A., Sanders, T., & van den Bergh, H. (2014). Effect of genre on the generalizability of writing scores. Language Testing, 32(1), 83-100. https: / / doi.org/ 10. 1177/ 0265532214542994 Brinker, K., Cölfen, H., & Pappert, S. (2018). Linguistische Textanalyse: Eine Einführung in Grundbegriffe und Methoden (8th ed.). Berlin: Erich Schmidt Verlag. Brock, R. (2015). Handbuch Writing (2014 und 2015). Kommentierte Schreibperformanzen 2014 und 2015: Gesamtbeurteilungen. Richtlinien zur Bewertung von Schreibleistungen (6. und 7. Schulstufe). Salzburg: Bundesinstitut für Bildungsforschung, Innovation & Entwicklung des österreichischen Schulwesens (BIFIE). https: / / www.bifie.at/ node/ 3 148 Bruning, R., & Horn, C. (2000). Developing motivation to write. Educational Psychologist, 35(1), 25-37. https: / / doi.org/ 10.1207/ S15326985EP3501_4 Bulut, P. (2017). The effect of primary school students' writing attitudes and writing self-efficacy beliefs on their summary writing achievement. International Electronic Journal of Elementary Education, 10(2), 281-285. https: / / doi.org/ 10.26822/ iejee.201723 612 Butler, Y. G., & Zeng, W. (2014). Young learners‘ storytelling in their first and foreign lan‐ guages. In J. Connor-Linton & L. W. Amoroso (Eds.), Measured language. Quantitative studies of acquisition, assessment, and variation (pp. 79-94). Washington: Georgetown University Press. Cameron, L. (2001). Teaching languages to young learners. Cambridge: Cambridge Uni‐ versity Press. Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1-47. https: / / doi.org/ 10.1093 / applin/ I.1.1 Charlton, C., Rasbash, J., Browne, W. J., Healy, M., & Cameron, B. (2020). MLwiN (Version 3.05) [Computer software]. University of Bristol: Centre for Multilevel Modelling. http: / / www.bristol.ac.uk/ cmm/ software/ mlwin/ Cho, Y., & So, Y. (2014). Construct-irrelevant factors influencing young English as a foreign language (EFL) learners’ perceptions of test task difficulty. Princeton: Educational Testing Service. https: / / www.ets.org/ research/ policy_research_reports/ publications/ report/ 2014/ jsfe Cizek, G. J. (Ed.) (2012). Setting performance standards. Foundations, methods, and innova‐ tions (2nd ed.). New York: Routledge. Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Thousand Oak: Sage Publications. 251 References Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Lawrence Erlbaum Associates. Coley, D. A., Greeves, R., & Saxby, B. K. (2007). The effect of low ventilation rates on the cognitive function of a primary school class. International Journal of Ventilation, 6(2), 107-112. https: / / doi.org/ 10.1080/ 14733315.2007.11683770 Connelly, V., Gee, D., & Walsh, E. (2007). A comparison of keyboarded and handwritten compositions and the relationship with transcription speed. British Journal of Educa‐ tional Psychology, 77(2), 479-492. https: / / doi.org/ 10.1348/ 000709906X116768 Council of Europe (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press. D-EDK (2015). Lehrplan 21. Luzern: Deutschschweizer Erziehungsdirektoren-Konferenz. https: / / v-fe.lehrplan.ch/ downloads.php D-EDK (2016). Lehrplan 21 - von der D-EDK Plenarversammlung am 31. 10. 2014 zur Einführung in den Kantonen freigegebene Vorlage. Bereinigte Fassung vom 29. 02. 2016. Luzern: Deutschschweizer Erziehungsdirektoren-Konferenz. https: / / v-fe.lehrplan.ch / downloads.php Daase, A., Hinrichs, B., & Settinieri, J. (2014). Befragung. In J. Settinieri, S. Demirkaya, A. Feldmeier, N. Gültekin-Karakoç, & C. Riemer (Eds.), Empirische Forschungsme‐ thoden für Deutsch als Fremd- und Zweitsprache. Eine Einführung (pp. 103-122). Paderborn: Ferdinand Schöningh. de Beaugrande, R.-A., & Dressler, W. U. (2013). Introduction to text linguistics. London: Routledge. De La Paz, S. (1999). Composing via dictation and speech recognition systems: Compen‐ satory technology for students with learning disabilities. Learning Disability Quarterly, 22(3), 173-182. https: / / doi.org/ 10.2307/ 1511284 de Larios, J. R., Manchón, R. M., & Murphy, L. (2006). Generating text in native and foreign language writing: A temporal analysis of problem-solving formulation processes. The Modern Language Journal, 90(1), 100-114. https: / / doi.org/ 10.1111/ j.1540-4781.2006.00 387.x de Oliveira, L., & Silva, T. (Eds.). (2016). Second language writing in elementary classrooms: Instructional issues, content-area writing and teacher education. Basingstoke: Palgrave Macmillan. DeGroff, L.-J. C. (1987). The influence of prior knowledge on writing, conferencing, and revising. The Elementary School Journal, 88(2), 105-118. https: / / www.jstor.org/ stable / 1002036 Donovan, C. A., & Smolkin, L. B. (2002). Children's genre knowledge: An examination of K-5 students' performance on multiple tasks providing differing levels of scaffolding. Reading Research Quarterly, 37(4), 428-465. https: / / doi.org/ 10.1598/ rrq.37.4.5 252 References Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3), 241-252. https: / / doi.org/ 10.1080/ 00401706.1964.10490181 EDK (2004a). HARMOS Zielsetzungen und Konzeption. Bern: Schweizerische Konferenz der kantonalen Erziehungsdirektoren. https: / / edudoc.ch/ record/ 33485 EDK (2004b). Sprachenunterricht in der obligatorischen Schule: Strategie der EDK und Ar‐ beitsplan für die gesamtschweizerische Koordination. Bern: Schweizerische Konferenz der kantonalen Erziehungsdirektoren. https: / / edudoc.ch/ record/ 30008 EDK (2011). Grundkompetenzen für die Fremdsprachen: Nationale Bildungsstandards. Bern: Schweizerische Konferenz der kantonalen Erziehungsdirektoren. https: / / edudo c.ch/ record/ 96780 EDK (2012). Koordination des Sprachenunterrichts in der Schweiz: Aktueller Stand - Entwicklungen - Ausblick. Bern: Schweizerische Konferenz der kantonalen Erzie‐ hungsdirektoren. https: / / edudoc.ch/ record/ 106282 EDK (2018). Faktenblatt Fremdsprachenunterricht in der obligatorischen Schule. Bern: Schweizerische Konferenz der kantonalen Erziehungsdirektoren. http: / / www.edk.ch / dyn/ 11911.php Egli Cuenat, M. (2016). Schreiben in drei Sprachen: Sprachenübergreifender Erwerb von Textkompetenz im schulischen Kontext. Bulletin VALS-ASLA, 103, 57-78. http: / / doc .rero.ch/ record/ 278448 Engber, C. A. (1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4(2), 139-155. https: / / doi.org/ 10.1 016/ 1060-3743(95)90004-7 Feez, S. (2002). Heritage and innovation in second language education. In A. M. Johns (Ed.), Genre in the classroom. Multiple perspectives (pp. 43-72). London: Lawrence Erlbaum Associates. Feilke, H. (2014). Begriff und Bedingungen literaler Kompetenz. In H. Feilke & T. Pohl (Eds.), Deutschunterricht in Theorie und Praxis. Schriftlicher Sprachgebrauch. Texte verfassen (pp. 33-53). Baltmannsweiler: Schneider Verlag Hohengehren. Ferreiro, E., & Pontecorvo, C. (1999). Managing the written text: The beginning of punctuation in children's writing. Learning and Instruction, 9(6), 543-564. https: / / doi .org/ 10.1016/ S0959-4752(99)00006-7 Furger, J., & Lindauer, T. (2013). Schreiben in einer interaktiven Web-Umgebung. Schreib‐ kompetenzen und Schreibmotivationen modellieren und erfassen. In C. Rosebrock & A. Bertschi-Kaufmann (Eds.), Literalität erfassen: Bildungspolitisch, kulturell, individ‐ uell (pp. 40-54). Weinheim: Beltz Juventa. Gnos, C. (2012). Englischkompetenzen am Ende der 6. Klasse: Überprüfung der Lernziele. Sprachstandserhebung im Kanton Luzern. Luzern: Dienststelle Volksschulbildung des Kantons Luzern. https: / / edudoc.ch/ record/ 113917/ 253 References Graham, S., McKeown, D., Kiuhara, S., & Harris, K. R. (2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational Psychology, 104(4), 879-896. https: / / doi.org/ 10.1037/ a0029185 Graham, S., & Sandmel, K. (2011). The process writing approach: A meta-analysis. The Journal of Educational Research, 104(6), 396-407. https: / / doi.org/ 10.1080/ 00220671.20 10.488703 Griva, E., & Chostelidou, D. (2013). Writing skills and strategies of bilingual immigrant students learning Greek as a second language and English as a foreign language. Reading & Writing, 4(1), 1-9. https: / / doi.org/ 10.4102/ rw.v4i1.31 Griva, E., Tsakiridou, H., & Nihoritou, I. (2009). A study of FL composing process and writing strategies employed by young learners. In M. Nikolov (Ed.), Early learning of modern foreign languages. Processes and outcomes (pp. 132-148). Bristol: Multilingual Matters. Grotjahn, R. (2017). Testkonstrukt und Testspezifikationen. In B. Akukwe, R. Grotjahn, & S. Schipolowski (Eds.), Schreibkompetenzen in der Fremdsprache. Aufgabengestaltung, kriterienorientierte Bewertung und Feedback (pp. 71-116). Tübingen: Narr Francke Attempto Verlag. Grotjahn, R., & Kleppin, K. (2017a). Entwicklung von Testaufgaben zum Schreiben. In B. Akukwe, R. Grotjahn, & S. Schipolowski (Eds.), Schreibkompetenzen in der Fremdsprache. Aufgabengestaltung, kriterienorientierte Bewertung und Feedback (pp. 159-180). Tübingen: Narr Francke Attempto. Grotjahn, R., & Kleppin, K. (2017b). Gütekriterien bei der Evaluation von Schreibkom‐ petenzen. In B. Akukwe, R. Grotjahn, & S. Schipolowski (Eds.), Schreibkompetenzen in der Fremdsprache. Aufgabengestaltung, kriterienorientierte Bewertung und Feedback (pp. 41-70). Tübingen: Narr Francke Attempto. Grotjahn, R., & Kleppin, K. (2017c). Kriteriale Evaluation von Schreibkompetenzen. In B. Akukwe, R. Grotjahn, & S. Schipolowski (Eds.), Schreibkompetenzen in der Fremdsprache. Aufgabengestaltung, kriterienorientierte Bewertung und Feedback (pp. 117-157). Tübingen: Narr Francke Attempto. Hall, N., & Sing, S. (2011). Sevento nine-year-olds' understandings of speech marks: Some issues and problems. Visible Language, 45(1-2), 69-92. https: / / visiblelanguage. herokuapp.com/ issue/ 152/ article/ 844 Hallet, W. (2016). Genres im fremdsprachlichen und bilingualen Unterricht. Formen und Muster der sprachlichen Interaktion. Seelze: Klett, Kallmeyer. Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman. Hambleton, R. K., Jaeger, R. M., Plake, B. S., & Mills, C. (2000). Setting performance standards on complex educational assessments. Applied Psychological Measurement, 24(4), 355-366. https: / / doi.org/ 10.1177/ 01466210022031804 254 References Harsch, C., & Martin, G. (2012). Adapting CEF-descriptors for rating purposes: Validation by a combined rater training and scale revision approach. Assessing Writing, 17(4), 228-250. https: / / doi.org/ 10.1016/ j.asw.2012.06.003 Harsch, C., Schröder, K., & Neumann, A. (2008). Schreiben Englisch. In DESI-Konsor‐ tium (Ed.), Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie (pp. 139-148). Weinheim: Beltz. Hasselgreen, A., Kaledaite, V., Maldonado Martín, N., & Pizorn, K. (2012). Assessment of young learner literacy linked to the Common European Framework of Reference for Languages. Strasbourg: Council of Europe Publishing. http: / / www.ecml.at/ tabid/ 277/ PublicationID/ 63/ Default.aspx Hasselgreen, A., & Sundet, K. T. (2017). Introducing the CORYL corpus: What it is and how we can use it to shed light on learner language. Bergen Language and Linguistics Studies, 7, 197-215. https: / / doi.org/ 10.15845/ bells.v7i0.1107 Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29(3), 369-388. https: / / doi.org/ 10.1177/ 0741088312451260 Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30). Hillsdale: Lawrence Erlbaum Associates. Hayes, J. R., & Olinghouse, N. G. (2015). Can cognitive writing models inform the design of the common core state standards? The Elementary School Journal, 115(4), 480-497. https: / / doi.org/ 10.1086/ 681909 Heck, R. H., Thomas, S. L., & Tabata, L. N. (2010). Multilevel and longitudinal modeling with IBM SPSS. New York: Routledge. Husfeldt, V., & Bader-Lehmann, U. (2009). Englisch an der Primarschule: Erfahrungen aus der Schweiz. In G. Engel, B. Groot-Wilken, & E. Thürmann (Eds.), Englisch in der Primarstufe - Chancen und Herausforderungen. Evaluation und Erfahrungen aus der Praxis (pp. 111-123). Berlin: Cornelsen Verlag. Hüttis-Graff, P., & Merklinger, D. (2010). Ohne Buchstaben Texte schreiben. Ein Hörspiel für Kinder als Zugang zu Schriftlichkeit. In D. Grenz (Ed.), Kinder- und Jugendliter‐ atur. Theorie, Geschichte, Didaktik (pp. 179-198). Baltmannsweiler: Schneider-Verlag Hohengehren. Hyland, K. (2004). Genre and second language writing. Ann Arbor: University of Michigan. Hyland, K. (2007). Genre pedagogy: Language, literacy and L2 writing instruction. Journal of Second Language Writing, 16(3), 148-164. https: / / doi.org/ 10.1016/ j.jslw.2007.07.005 Hyland, K. (2011). Learning to write. In R. M. Manchón (Ed.), Learning-to-write and writing-to-learn in an additional language (pp. 17-35). Amsterdam: John Benjamins. Hyland, K. (2019). Second language writing (2nd ed.). Cambridge: Cambridge University Press. 255 References Hymes, D. H. (1972). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics. Selected readings (pp. 269-293). Harmondsworth: Penguin. Jarvis, S. (2002). Short texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19(1), 57-84. https: / / doi.org/ 10.1191/ 0265532202lt220oa Kingston, N. M., & Tiemann, G. C. (2012). Setting performance standards using the Body of Work method. In G. J. Cizek (Ed.), Setting performance standards. Foundations, methods, and innovations (2nd ed., pp. 194-216). New York: Routledge. Klassen, R. (2002). Writing in early adolescence: A review of the role of self-efficacy beliefs. Educational Psychology Review, 14(2), 173-203. https: / / doi.org/ 10.1023/ A: 1014 626805572 Klein, P. D., & Leacock, T. L. (2012). Distributed cognition as a framework for under‐ standing writing. In V. W. Berninger (Ed.), Past, present, and future contributions of cognitive writing research to cognitive psychology (pp. 133-152). New York: Psychology Press. Knoch, U. (2009). Diagnostic writing assessment. The development and validation of a rating scale. Frankfurt am Main: Peter Lang. Konsortium HarmoS Fremdsprachen (2009). Fremdsprachen. Wissenschaftlicher Kurz‐ bericht und Kompetenzmodell. Provisorische Fassung (vor Verabschiedung der Stand‐ ards). Fribourg: Université de Fribourg. https: / / edudoc.ch/ record/ 87025 Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163. https: / / doi.org/ 10.1016/ j.jcm.2016.02.012 Koster, M., Tribushinina, E., de Jong, P. F., & van den Bergh, H. (2015). Teaching children to write: A meta-analysis of writing intervention research. Journal of Writing Research, 7(2), 249-274. https: / / doi.org/ 10.17239/ jowr-2015.07.02.2 Kreis, A., Williner, M., & Maeder, C. (2014). Englischunterricht in der Primarschule des Kantons Thurgau. Schlussbericht der Evaluation. Kreuzlingen: Pädagogische Hoch‐ schule Thurgau. https: / / edudoc.ch/ record/ 111928 Labov, W., & Waletzky, J. (1997). Narrative analysis: Oral versions of personal experience. Journal of Narrative and Life History, 7(1-4), 3-38. https: / / doi.org/ 10.1075/ jnlh.7.02nar Lasagabaster, D., & Doiz, A. (2003). Maturational constraints on foreign language written production. In M. d. P. García Mayo & M. L. G. Lecumberri (Eds.), Age and the acquisition of English as a foreign language (pp. 136-160). Clevedon: Multilingual Matters. Latcheva, R., & Davidov, E. (2019). Skalen und Indizes. In N. Baur & J. Blasius (Eds.), Hand‐ buch Methoden der empirischen Sozialforschung (pp. 893-905). Wiesbaden: Springer Fachmedien. https: / / doi.org/ 10.1007/ 978-3-658-21308-4_62 256 References Lee, I., & Wong, K. (2014). Bringing innovation to EFL writing: The case of a primary school in Hong Kong. Asia-Pacific Education Researcher, 23(1), 159-163. https: / / doi.or g/ 10.1007/ s40299-013-0149-y Leki, I., Cumming, A., & Silva, T. (2008). A synthesis of research on second language writing in English. New York: Routledge. Leuders, T. (2015). Empirische Forschung in der Fachdidaktik - eine Herausforderung für die Nachwuchsqualifizierung. Paper presented at the Tagung Fachdidaktiken vom 22. Januar 2015, Bern. Li, X., Chu, S. K. W., & Ki, W. W. (2014). The effects of a wiki-based collaborative process writing pedagogy on writing ability and attitudes among upper primary school students in Mainland China. Computers & Education, 77, 151-169. https: / / doi.org/ 10. 1016/ j.compedu.2014.04.019 Lienert, G. A., & Raatz, U. (1998). Testaufbau und Testanalyse (6th ed.). Weinheim: Beltz PsychologieVerlagsUnion. Linacre, J. M. (2018). A user’s guide to FACETS: Rasch-model computer programs [Software manual]. Chicago: Winsteps.com. https: / / www.winsteps.com/ manuals.htm Lindgren, E., & Stevenson, M. (2013). Interactional resources in the letters of young writers in Swedish and English. Journal of Second Language Writing, 22(4), 390-405. https: / / doi.org/ 10.1016/ j.jslw.2013.09.001 Llach, M. d. P. A., & Gómez, A. B. (2007). Children's characteristics in vocabulary acquisition and use in the written production. Revista Espanola de Linguistica Aplicada, 20, 9-26. https: / / dialnet.unirioja.es/ servlet/ articulo? codigo=2514258 Loder Buechel, L. (2015). Young learner writing performance in Swiss elementary schools - which teacher variables matter? RELC Journal, 46(3), 275-292. https: / / doi. org/ 10.1177/ 0033688215598437 Loomis, S. C., & Bourque, M. L. (2001). From tradition to innovation: Standard setting on the National Assessment of Educational Progress. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 175-218). Mahwah: Erlbaum. Maiello, C. (2011). Verhaltenswissenschaftliche Forschung für Einsteiger: Methodenlehre, Statistik und computergestützte Datenauswertung (2nd ed.). Landau: Verlag Empirische Pädagogik. McCutchen, D. (1986). Domain knowledge and linguistic knowledge in the development of writing ability. Journal of Memory and Language, 25(4), 431-444. https: / / doi.org/ 10 .1016/ 0749-596X(86)90036-7 McCutchen, D. (2006). Cognitive factors in the development of children's writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 115-130). New York: Guilford Press. 257 References McCutchen, D., Francis, M., & Kerr, S. (1997). Revising for meaning: Effects of knowledge and strategy. Journal of Educational Psychology, 89(4), 667-676. https: / / doi.org/ 10.103 7/ 0022-0663.89.4.667 McKay, P. (2006). Assessing young language learners. Cambridge: Cambridge University Press. McNeish, D. (2017). Small sample methods for multilevel modeling: A colloquial elucida‐ tion of REML and the Kenward-Roger correction. Multivariate Behavioral Research, 52(5), 661-670. https: / / doi.org/ 10.1080/ 00273171.2017.1344538 Merklinger, D. (2010). 'Lernendes Schreiben' am Übergang von Mündlichkeit zu Schrift‐ lichkeit. In T. Pohl & T. Steinhoff (Eds.), Textformen als Lernformen (pp. 117-142). Duisburg: Gilles & Francke. Meyer, P. G. (2008). Descriptive English linguistics. An introduction (4th ed.). Tübingen: Narr Francke Attempto. Midgette, E., Haria, P., & MacArthur, C. (2008). The effects of content and audience awareness goals for revision on the persuasive essays of fifthand eighth-grade students. Reading and Writing, 21(1), 131-151. https: / / doi.org/ 10.1007/ s11145-007-906 7-9 Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-cen‐ tered design. ETS Research Report Series, i-29. https: / / doi.org/ 10.1002/ j.2333-8504.200 3.tb01908.x Mosenthal, P. B., Conley, M. W., Colella, A., & Davidson-Mosenthal, R. (1985). The influence of prior knowledge and teacher lesson structure on children's production of narratives. The Elementary School Journal, 85(5), 621-634. https: / / doi.org/ 10.1086/ 461 425 Nolen, S. B. (2007). Young children's motivation to read and write: Development in social contexts. Cognition and Instruction, 25(2-3), 219-270. https: / / doi.org/ 10.1080/ 0737000 0701301174 Norris, E., Mokhtari, K., & Reichard, C. (1998). Children’s use of drawing as a pre‐writing strategy. Journal of Research in Reading, 21(1), 69-74. https: / / doi.org/ 10.1111/ 1467-98 17.00043 Olinghouse, N. G., & Graham, S. (2009). The relationship between the discourse knowl‐ edge and the writing performance of elementary-grade students. Journal of Educa‐ tional Psychology, 101(1), 37-50. https: / / doi.org/ 10.1037/ a0013462 Olinghouse, N. G., Graham, S., & Gillespie, A. (2015). The relationship of discourse and topic knowledge to fifth graders’ writing performance. Journal of Educational Psychology, 107(2), 391-406. https: / / doi.org/ 10.1037/ a0037549 Olinghouse, N. G., & Wilson, J. (2013). The relationship between vocabulary and writing quality in three genres. Reading and Writing, 26(1), 45-65. https: / / doi.org/ 10.1007/ s11 145-012-9392-5 258 References Olsson, E. (2011). "Everything I read on the Internet is in English." On the impact of extramural English on Swedish 16-year-old pupils’ writing proficiency. Göteborg: Institutet för svenska som andraspråk och Institutionen för svenska språket. https: / / core.ac.uk/ download/ pdf/ 16334333.pdf. Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature. Reading & Writing Quarterly, 19(2), 139-158. https: / / doi.org/ 10.1080 / 10573560308222 Pajares, F., & Valiante, G. (1997). Influence of self-efficacy on elementary students' writing. The Journal of Educational Research, 90(6), 353-360. https: / / doi.org/ 10.1007/ s 11145-012-9392-5 Parr, J. M., & Timperley, H. S. (2010). Feedback to writing, assessment for teaching and learning and student progress. Assessing Writing, 15(2), 68-85. https: / / doi.org/ 10.101 6/ j.asw.2010.05.004 Philipp, M. (2015). Grundlagen der effektiven Schreibdidaktik. Baltmannsweiler: Schneider Verlag Hohengehren. Pinter, A. (2017). Teaching young language learners (2nd ed.). Oxford: Oxford University Press. Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 61(4), 993-1038. https: / / doi.org/ 10.1111/ j.1467-9 922.2011.00663.x Pon, L., & Bagarić Medve, V. (2017). Methoden zur Analyse und Bewertung von Kohärenz in schriftlichen Lernertexten. In K. Cergol Kovacevic & S. L. Udier (Eds.), Applied linguistics research and methodology. Proceedings from the 2015 CALS conference (pp. 71-88). Frankfurt am Main: Peter Lang. https: / / doi.org/ 10.3726/ b10830 Porsch, R. (2010). Schreibkompetenzvermittlung im Englischunterricht in der Sekundar‐ stufe I. Empirische Analysen zu Leistungen, Einstellungen, Unterrichtsmethoden und Zusammenhängen von Leistungen in der Mutter- und Fremdsprache. Münster: Wax‐ mann. Porsch, R., & Köller, O. (2010). Standardbasiertes Testen von Schreibkompetenzen im Fach Englisch. In W. Bos, E. Klieme, & O. Köller (Eds.), Schulische Lerngelegenheiten und Kompetenzentwicklung. Festschrift für Jürgen Baumert (pp. 85-103). Münster: Waxmann. Raaen, G. F., & Guldal, T. M. (2012). Basic skills: Orthography and sentence complexity in written English from grades 7 to 10. In A. Hasselgreen, I. Drew, & B. Sørheim (Eds.), The young language learner. Research-based insights into teaching and learning (pp. 101-118). Bergen: Fagbokforlaget. Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2020). A user's guide to MLwiN [Software manual]. University of Bristol: Centre for Multilevel Modelling. http: / / ww w.bristol.ac.uk/ cmm/ software/ mlwin/ download/ manuals.html 259 References Read, J. C. (2006). A study of the usability of handwriting recognition for text entry by children. Interacting with Computers, 19(1), 57-69. https: / / doi.org/ 10.1016/ j.intcom.20 06.08.009 Richards, J., & Rodgers, T. (2014). Approaches and methods in language teaching (3rd ed.). Cambridge: Cambridge University Press. Richards, J. C., & Schmidt, R. (2010). Longman dictionary of language teaching and applied linguistics (4th ed.). Harlow: Pearson Education Limited. Rietdijk, S., Janssen, T., Van Weijen, D., Van den Bergh, H., & Rijlaarsdam, G. (2017). Improving writing in primary schools through a comprehensive writing program. Journal of Writing Research, 9(2), 173-225. https: / / doi.org/ 10.17239/ jowr-2017.09.02.04 Rijlaarsdam, G., & Van den Bergh, H. (2005). Effective learning and teaching of writing: Student involvement in the teaching of writing. In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Effective learning and teaching of writing. A handbook of writing in education (2nd ed., pp. 1-16). New York: Kluwer Academic. Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing process theory. A functional dynamic approach. In C. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 41-53). New York: The Guilford Press. Schipolowski, S., & Böhme, K. (2016). Assessment of writing ability in secondary education: Comparison of analytic and holistic scoring systems for use in large-scale assessments. L1 Educational Studies in Language and Literature, 16, 1-22. https: / / doi. org/ 10.17239/ L1ESLL-2016.16.01.03 Schoonen, R., van Gelderen, A., Stoel, R. D., Hulstijn, J., & de Glopper, K. (2011). Modeling the development of L1 and EFL writing proficiency of secondary school students. Language Learning, 61(1), 31-79. https: / / doi.org/ 10.1111/ j.1467-9922.2010.00590.x Schunk, D. H., & Swartz, C. W. (1991). Process goals and progress feedback: Effects on children's self-efficacy and skills. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago. Shell, D. F., Colvin, C., & Bruning, R. H. (1995). Self-efficacy, attribution, and outcome expectancy mechanisms in reading and writing achievement: Grade-level and ach‐ ievement-level differences. Journal of Educational Psychology, 87(3), 386-398. https: / / doi.org/ 10.1037/ 0022-0663.87.3.386 Sörqvist, P., Nöstl, A., & Halin, N. (2012). Disruption of writing processes by the semanticity of background speech. Scandinavian Journal of Psychology, 53(2), 97-102. https: / / doi.org/ 10.1111/ j.1467-9450.2011.00936.x Stæhr, L. S. (2008). Vocabulary size and the skills of listening, reading and writing. Language Learning Journal, 36(2), 139-152. https: / / doi.org/ 10.1080/ 0957173080238997 5 260 References Stein, N. L., & Policastro, M. (1984). The concept of a story: A comparison between child‐ ren’s and teachers’ viewpoints. In H. Mandl, N. L. Stein, & T. Trabasso (Eds.), Learning and comprehension of text (pp. 113-158). Hillsdale: Lawrence Erlbaum Associates. Südkamp, A., Kaiser, J., & Möller, J. (2012). Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology, 104(3), 743-762. https: / / doi.org/ 10.1037/ a0027627 Sundqvist, P. (2009). The impact of spare time activities on students' English lan‐ guage skills. In S. Granath, B. Bihl, & E. Wennö (Eds.), Vägar till språk och litteratur (pp. 63-76). Karlstad: Karlstads Universitet. http: / / kau.diva-portal.org/ smash/ record. jsf ? pid=diva2%3A292259&dswid=4688. Sylvén, L. K., & Sundqvist, P. (2012). Gaming as extramural English L2 learning and L2 proficiency among young learners. ReCALL, 24(3), 302-321. https: / / doi.org/ 10.1017/ S095834401200016X Szabo, T. (2016). Collated representative samples of descriptors of language competences developed for young learners aged 11-15 years. Strasbourg: Council of Europe. http: / / www.coe.int/ en/ web/ portfolio/ overview-of-cefr-related-scales Szabo, T. (2018). Collated representative samples of descriptors of language competences developed for young learners. Volume 2: Ages 11-15. Strasbourg: Council of Eu‐ rope. https: / / rm.coe.int/ collated-representative-samples-descriptors-young-learnersvolume-2-ag/ 16808b1689 Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson. Tankersley, K. (2007). Tests that teach: Using standardized tests to improve instruction. Alexandria: ASCD Association for Supervision and Curriculum Development. Tannenbaum, R. J., & Baron, P. A. (2010). Mapping TOEIC® test scores to the STANAG 6001 language proficiency levels. Princeton: Educational Testing Service. https: / / www .ets.org/ Media/ Research/ pdf/ RM-10-11.pdf Tannenbaum, R. J., & Cho, Y. (2014). Critical factors to consider in evaluating standard-setting studies to map language test scores to frameworks of language proficiency. Language Assessment Quarterly, 11(3), 233-249. http: / / dx.doi.org/ 10.1080 / 15434303.2013.869815 Tracy, B., Reid, R., & Graham, S. (2009). Teaching young students strategies for planning and drafting stories: The impact of self-regulated strategy development. The Journal of Educational Research, 102(5), 323-332. https: / / doi.org/ 10.3200/ JOER.102.5.323-332 Troia, G., Harbaugh, A., Shankland, R., Wolbers, K., & Lawrence, A. (2013). Relationships between writing motivation, writing activity, and writing performance: Effects of grade, sex, and ability. Reading and Writing, 26(1), 17-44. https: / / doi.org/ 10.1007/ s11 145-012-9379-2 261 References Tsiriotakis, I. K., Vassilaki, E., Spantidakis, I., & Stavrou, N. A. M. (2017). The examination of the effects of writing strategy-based procedural facilitative environments on students' English foreign language writing anxiety levels. Frontiers in Psychology, 7, 1-14. https: / / doi.org/ 10.3389/ fpsyg.2016.02074 UNESCO (2012). International standard classification of education. ISCED 2011. Montreal: UNESCO Institute for Statistics. http: / / uis.unesco.org/ sites/ default/ files/ documents/ international-standard-classification-of-education-isced-2011-en.pdf van der Hoeven, J. (1997). Children's composing: A study into the relationships between writing processes, text quality, and cognitive and linguistic skills. Amsterdam: Rodopi. Vögelin, C., Jansen, T., Keller, S. D., & Möller, J. (2018). The impact of vocabulary and spelling on judgments of ESL essays: An analysis of teacher comments. The Language Learning Journal, 1-17. https: / / doi.org/ 10.1080/ 09571736.2018.1522662 Vogt, S., & Bader, U. (2017). Vorläufiger Forschungsbericht des Forschungs- und Entwick‐ lungsprojektes Schreiben im Englischunterricht an der Primarschule (SEPS). [Unpub‐ lished report]. Brugg-Windisch: Pädagogische Hochschule FHNW. VSA (2016). Kompetenzorientiert fördern und beurteilen. Solothurn: Volksschulamt Kanton Solothurn. https: / / www.so.ch/ fileadmin/ internet/ dbk/ dbk-vsa/ Foerdern/ Sch ullaufbahn/ 2016_brosch%C3%BCre_kompetenzorientiert_beurteilen.pdf Wagner, W., Helmke, A., & Rösner, E. (2009). Deutsch Englisch Schülerleistungen Interna‐ tional. Dokumentation der Erhebungsinstrumente für Schülerinnen und Schüler, Eltern und Lehrkräfte. Frankfurt am Main: Deutsches Institut für Internationale Pädagogi‐ sche Forschung. http: / / www.pedocs.de/ volltexte/ 2010/ 3252. Wang, X. (2014). The relationship between lexical diversity and EFL writing proficiency. University of Sydney Papers in TESOL, 9, 65-88. http: / / faculty.edfac.usyd.edu.au/ proj ects/ usp_in_tesol/ volume09.htm Wargocki, P., Wyon, D. P., Matysiak, B., & Irgens, S. (2005). The effects of classroom air temperature and outdoor air supply rate on performance of school work by children. Proceedings of Indoor Air, 1(1), 368-372. https: / / www.isiaq.org/ docs/ PDFs/ 0368.pdf Weigle, S. C. (2002). Assessing writing (6th ed.). Cambridge: Cambridge University Press. Yarrow, F., & Topping, K. J. (2001). Collaborative writing: The effects of metacognitive prompting and structured peer interaction. British Journal of Educational Psychology, 71(2), 261-282. https: / / doi.org/ 10.1348/ 000709901158514 Yasuda, S. (2019). Children's meaning-making choices in EFL writing: The use of cohesive devices and interpersonal resources. System, 85, 1-15. https: / / doi.org/ 10.1016/ j.syste m.2019.102108 Yu, G. (2009). Lexical diversity in writing and speaking task performances. Applied Linguistics, 31(2), 236-259. https: / / doi.org/ 10.1093/ applin/ amp024 Zarei, A., & Khalili, T. (2017). A meta-analysis of the effectiveness of genre-based writing instruction in English as foreign language. Language Related Research, 8(5), 107-137. 262 References Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21(2), 179-217. https: / / doi.org/ 10.1207/ s15516709cog2102_3 Zhang, J., & Patel, V. L. (2006). Distributed cognition, representation, and affordance. Pragmatics & Cognition, 14(2), 333-341. https: / / doi.org/ 10.1075/ pc.14.2.12zha Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: A manual for setting standards of performance on educational and occupational tests. Princeton: Educational Testing Service. 263 References Appendices Appendix A Compilation of CEFR descriptors relevant for young foreign language writers The following CEFR descriptors were compiled for this study on the basis of the illustrative CEFR descriptors considered as relevant or partially relevant for young learners aged 11-15 in Szabo (2018). Table A1 is a compilation of de‐ scriptors for written communicative language activities, Table A2 for pragmatic, Table A3 for sociolinguistic and Table A4 for linguistic communicative language competences. Pre-A1 A1 A2 A2+ B1 O v e r a l l w r i t t e n p r o d u c t i o n Can give basic personal infor‐ mation in writing (e. g. name, address, nationality), per‐ haps with the use of a dictionary. Can give informa‐ tion in writing about matters of personal relevance (e. g. likes and dislikes, family, pets) using simple words and basic ex‐ pressions. Can write simple iso‐ lated phrases and sentences. Can write a series of simple phrases and sentences linked with simple connectors like ‘and,’ ‘but’ and ’because.’ Can write straightfor‐ ward connected texts on a range of familiar sub‐ jects within his / her field of interest, by linking a series of shorter discrete elements into a linear se‐ quence. C r e a t i v e w r i t i n g Can write simple phrases and sen‐ tences about them‐ selves and imaginary people, where they live and what they do. Can describe very simply what a room looks like. Can use simple words and phrases to describe certain ev‐ eryday objects (for example the colour of a car, whether it is big or small). Can write a series of simple phrases and sentences about their family, living conditions, educa‐ tional background, present or most recent job. Can write short, simple imag‐ inary biographies and simple poems about people. Can write diary entries that de‐ scribe activities (e. g. daily rou‐ tine, outings, sports, hobbies), people and places, using basic, concrete vocabulary and simple phrases and sentences with simple connectives like ‘and,’ ‘but’ and ‘because.’ Can write about ev‐ eryday aspects of his / her environ‐ ment e. g. people, places, a job or study experience in linked sentences. Can write very short, basic descrip‐ tions of events, past activities and per‐ sonal experiences. Can tell a simple story (e. g. about events on a holiday or about life in the distant future). Can write straightfor‐ ward, detailed descrip‐ tions on a range of fa‐ miliar subjects within his / her field of interest. Can write accounts of experiences, describing feelings and reactions in simple connected text. Can write a description of an event, a recent trip - real or imagined. Can narrate a story. 266 Appendices Pre-A1 A1 A2 A2+ B1 Can write an introduction to a story or continue a story, pro‐ vided he / she can consult a dic‐ tionary and references (e. g. tables of verb tenses in a coursebook). W r i t t e n r e p o r t s a n d e s s a y s Can write simple texts on familiar subjects of interest, linking sen‐ tences with connectors like ‘and,’ ‘because,’ or ‘then.’ Can give his / her impressions and opinions in writing about topics of personal interest (e. g. lifestyles and culture, stories), using basic everyday vocabulary and expres‐ sions. Can give his / her impressions and opinions in writing about topics of per‐ sonal interest (e. g. lifestyles and cul‐ ture, stories), using basic everyday vo‐ cabulary and ex‐ pressions. Can write very brief reports to a standard conventionalised format, which pass on routine factual information and state reasons for actions. Can present a topic in a short report or poster, using photographs and short blocks of text. O v e r a l l w r i t t e n i n t e r a c t i o n Can write short phrases to give basic informa‐ tion (e. g. name, address, family) on a form or in a note, with the use of a dic‐ tionary. Can ask for or pass on personal details in written form. Can write short, simple formulaic notes relating to matters in areas of immediate need. Can write personal let‐ ters and notes asking for or conveying simple in‐ formation of immediate relevance, getting across the point he / she feels to be important. 267 Appendix A Pre-A1 A1 A2 A2+ B1 C o r r e s p o n d e n c e Can write short phrases and sen‐ tences giving basic personal information with reference to a dictionary. Can write messages and online post‐ ings as a series of very short sen‐ tences about hobbies and likes / dislikes, using simple words and formulaic ex‐ pressions, with refer‐ ence to a dictionary. Can write a short, very simple message (e. g. a text message) to friends to give them a piece of in‐ formation or to ask them a question. Can write a short, simple postcard. Can write very simple personal letters expressing thanks and apology. Can write short, simple notes, emails and text messages (e. g. to send or reply to an invitation, to confirm or change an arrange‐ ment). Can write a short text in a greet‐ ings card (e. g. for someone’s birthday or to wish them a Happy New Year). Can convey personal information of a routine nature, for example in a short email or letter introducing him / herself. Can exchange in‐ formation by text message, e-mail or in short letters, re‐ sponding to ques‐ tions the other person had (e. g. about a new product or activity). Can write personal letters describing expe‐ riences, feelings and events in some detail. Can write basic emails / letters of a factual nature, for example to re‐ quest information or to ask for and give confir‐ mation. Can write a basic letter of application with limited supporting details. Table A1 CEFR descriptors for a selection of written communicative language activities relevant for young learners aged 11-15 based on Szabo (2018), levels pre-A1 to B1 268 Appendices Pre-A1 A1 A2 A2+ B1 C o h e r e n c e a n d c o h e s i o n Can link words or groups of words with very basic linear connectors like 'and' or 'then'. Can link groups of words with simple connectors like ’and,’ ‘but’ and ‘because.’ Can use the most frequently occur‐ ring connectors to link simple sen‐ tences in order to tell a story or de‐ scribe something as a simple list of points. Can link a series of shorter, discrete simple elements into a connected, linear sequence of points. Can form longer sentences and link them together using a limited number of cohe‐ sive devices, e. g. in a story. Can make simple, logical paragraph breaks in a longer text. T h e m a t i c d e v e l o p m e n t Can give an ex‐ ample of some‐ thing in a very simple text using ‘like’ or ‘for ex‐ ample.’ Can tell a story or describe some‐ thing in a simple list of points. Shows awareness of the conventional structure of the text type concerned, when communicating his / her ideas. Can reasonably fluently relate a straight‐ forward narrative or description as a linear sequence of points. Can briefly give reasons and explana‐ tions for opinions. P r o p o s i t i o n a l p r e c i s i o n Can commu‐ nicate very basic infor‐ mation about personal de‐ tails in a simple way. Can communicate basic information about per‐ sonal details and needs of a concrete type in a simple way. Can communicate what he / she wants to say in a simple and direct exchange of limited information on familiar and routine matters, but in other situations he / she generally has to compromise the message. Can convey simple, straightforward in‐ formation of immediate relevance, get‐ ting across which point he / she feels is most important. Can express the main point he / she wants to make comprehensibly. 269 Appendix A Pre-A1 A1 A2 A2+ B1 F l e x i b i l i t y Can expand learned phrases through simple re-combinations of their ele‐ ments. Can adapt well-rehearsed memorised simple phrases to particular circum‐ stances through limited lexical substi‐ tution. Can adapt his / her expression to deal with less routine, even difficult, situa‐ tions. Can exploit a wide range of simple lan‐ guage flexibly to express much of what he / she wants. Table A2 CEFR descriptors for pragmatic communicative language competences relevant for young learners aged 11-15 based on Szabo (2018), levels pre-A1 to B1 270 Appendices Pre-A1 A1 A2 A2+ B1 S o c i o l i n g u i s t i c a p p r o p r i a t e n e s s Can establish basic so‐ cial contact by using the simplest everyday polite forms of: greetings and farewells; introductions; saying please, thank you, sorry etc. Can handle very short social ex‐ changes, using ev‐ eryday polite forms of greeting and address. Can make and respond to invitations, suggestions, apol‐ ogies etc. Can perform and respond to basic language func‐ tions, such as in‐ formation ex‐ change and requests and ex‐ press opinions and attitudes in a simple way. Can socialise simply but effec‐ tively using the simplest common expressions and following basic routines. Can perform and respond to a wide range of language functions, using their most common exponents in a neutral register Is aware of the salient politeness conven‐ tions and acts appropriately Is aware of, and looks out for signs of, the most significant differences between the customs, usages, attitudes, values and beliefs prevalent in the community concerned and those of his or her own community. Table A3 CEFR descriptors for sociolinguistic communicative language competence relevant for young learners aged 11-15 based on Szabo (2018), levels pre-A1 to B1 271 Appendix A Pre-A1 A1 A2 A2+ B1 G e n e r a l l i n g u i s t i c r a n g e Can use iso‐ lated words and basic ex‐ pressions in order to give simple infor‐ mation about him / herself. Has a very basic range of simple expressions about personal details and needs of a concrete type. Can use some basic struc‐ tures in one-clause sen‐ tences with some omis‐ sion or reduction of elements. Can produce brief ev‐ eryday expressions in order to satisfy simple needs of a concrete type: personal details, daily rou‐ tines, wants and needs, re‐ quests for information. Can use basic sentence patterns and communicate with memorised phrases, groups of a few words and formulae about them‐ selves and other people, what they do, places, pos‐ sessions etc. Has a limited repertoire of short memorised phrases covering predictable sur‐ vival situations; frequent breakdowns and misun‐ derstandings occur in non-routine situations. Has a repertoire of basic language, which enables him / her to deal with everyday situations with predictable content, though he / she will gen‐ erally have to compro‐ mise the message and search for words. Has enough language to get by, with sufficient vocabulary to express him / herself with some hesitation and circumlo‐ cutions on topics such as family, hobbies and interests, work, travel, and current events, but lexical limitations cause repetition and even diffi‐ culty with formulation at times. V o c a b u l a r y r a n g e Has a basic vocabulary repertoire of words and phrases related to partic‐ ular concrete situations. Has a sufficient vocabu‐ lary for the expression of basic communicative needs. Has sufficient vocabulary to conduct routine, ev‐ eryday transactions in‐ volving familiar situa‐ tions and topics. Has a good command of a range of vocabulary re‐ lated to familiar topics and everyday situations. Has a sufficient vocabu‐ lary to express him / her‐ 272 Appendices Pre-A1 A1 A2 A2+ B1 Has sufficient vocabulary for coping with simple sur‐ vival needs. self with some circumlo‐ cutions on most topics pertinent to his / her ev‐ eryday life such as family, hobbies and interests, work, travel, and current events. V o c a b u l a r y c o n t r o l Can control a narrow repertoire dealing with concrete everyday needs. Uses a wide range of simple vocabulary appro‐ priately when talking about familiar topics. G r a m m a t i c a l a c c u r a c y Can employ very simple principles of word order in short state‐ ments. Shows only limited con‐ trol of a few simple gram‐ matical structures and sentence patterns in a learnt repertoire. Uses some simple structures correctly, but still system‐ atically makes basic mistakes - for example tends to mix up tenses and forget to mark agreement; never‐ theless, it is usually clear what he / she is trying to say. Uses reasonably accu‐ rately a repertoire of fre‐ quently used ‘routines’ and patterns associated with more predictable situations. O r t h o g r a p h i c c o n t r o l Can copy familiar words and short phrases e. g. simple signs or instruc‐ tions, names of everyday objects, names of shops and set phrases used reg‐ ularly. Can copy short sentences on everyday subjects - e. g. directions how to get somewhere. Can write with reasonable phonetic accuracy (but not necessarily fully standard spelling) short words that are in his / her oral vocabulary. Can produce contin‐ uous writing, which is generally intelligible throughout. Spelling, punctuation and layout are accurate enough to be followed most of the time. 273 Appendix A Pre-A1 A1 A2 A2+ B1 Can spell his / her ad‐ dress, nationality and other personal details. Can use basic punctua‐ tion (e. g. full stops, ques‐ tion marks). Table A4 CEFR descriptors for linguistic communicative language competences relevant for young learners aged 11-15 based on Szabo (2018), levels pre-A1 to B1 274 Appendices Appendix B Learner questionnaire (in German) 275 Appendix B 276 Appendices 277 Appendix B 278 Appendices 279 Appendix B 280 Appendices 281 Appendix B Appendix C Scale and index documentation: learner questionnaire Scale Self-efficacy in writing in English Sources Adapted from Shell et al. (1995, p. 388) and Wagner et al. (2009, p. 59) v_3.1 Ich weiss, dass ich gute englische Sätze schreiben kann. v_3.3 Wenn ich mir beim Schreiben auf Englisch Mühe gebe, dann kann ich es auch. v_3.5 Ich weiss, dass ich auf Englisch so schreiben kann, dass der Leser es versteht. v_3.7 (excluded) Ich arbeite auch dann weiter, wenn das Schreiben auf Englisch mühsam ist. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Scale characteristics min max n M SD r it α if item deleted v_3.1 1 4 319 2.73 0.74 0.64 0.58 v_3.3 1 4 318 3.30 0.67 0.51 0.66 v_3.5 1 4 316 3.10 0.72 0.53 0.65 v_3.7 (excluded) 1 4 316 3.22 0.78 0.39 0.74 Scale n = 311 α = 0.72 Table C1 Scale for self-efficacy in writing in English 282 Appendices Scale Self-efficacy in learning English Source Adapted from Wagner et al. (2009, p. 59) v_2.1 Ich weiss, dass ich im Englisch gute Leistungen erbringen kann. v_2.3 Wenn ich mir im Englisch Mühe gebe, dann kann ich es auch. v_2.5 (excluded) Ich arbeite im Englisch auch dann weiter, wenn die Aufgabe schwierig ist. v_2.7 Wenn ich mir im Englisch Mühe gebe, bekomme ich auch gute Noten. v_2.9 Ich weiss, dass ich das, was wir im Englischunterricht machen, gut kann. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Scale characteristics min max n M SD r it α if item deleted v_2.1 1 4 320 3.33 0.72 0.64 0.69 v_2.3 1 4 319 3.66 0.56 0.57 0.72 v_2.5 (excluded) 1 4 320 3.26 0.72 0.37 0.79 v_2.7 1 4 320 3.63 0.55 0.53 0.73 v_2.9 1 4 318 3.15 0.67 0.62 0.69 Scale n = 317 α = 0.77 Table C2 Scale for self-efficacy in learning English 283 Appendix C Index Extra-curricular use of English Sources Adapted from Wagner et al. (2009, p. 39) and extended with items from Sylvén and Sundqvist (2012, p. 321) Englisch zu Hause und in der Freizeit v_1.1 Wie oft schaust du TV, Filme oder Videos auf Englisch? v_1.2 Wie oft hörst du dir englische Musik oder Hörbücher an? v_1.3 Wie oft liest du englische Texte (z.B. Bücher, Zeitschriften oder Texte im Internet)? v_1.4 Wie oft schreibst du selber etwas auf Englisch (z.B. Ge‐ schichten, Briefe, WhatsApp, E-Mail, SMS, Chats)? v_1.5 Wie oft sprechen deine Eltern zu Hause Englisch? v_1.6 Wie oft sprichst du zu Hause Englisch? v_1.7 Wie oft habt ihr mit jemandem Kontakt, der Englisch spricht? v_1.8 Wie oft sprecht ihr Englisch, wenn ihr gemeinsam etwas macht (z.B. Spiele, Lieder, Geschichten vorlesen, Ausflüge, Ferien)? Response categories nie (0) selten, 1-2 Mal pro Jahr (1), gelegentlich, 1-2 Mal pro Monat (2), oft, 1-2 Mal pro Woche (3), jeden Tag (4) Variable Item values Index characteristics min max n M SD r it α if item deleted v_1.1 0 4 320 2.04 1.24 0.50 0.77 v_1.2 0 4 318 3.35 1.04 0.19 0.81 v_1.3 0 4 320 1.75 1.10 0.53 0.76 v_1.4 0 4 320 1.63 1.26 0.57 0.75 v_1.5 0 4 319 0.96 1.15 0.46 0.77 v_1.6 0 4 319 1.40 1.19 0.60 0.75 v_1.7 0 4 318 1.47 1.25 0.52 0.76 v_1.8 0 4 319 1.50 1.14 0.59 0.75 Index n = 314 α = 0.79 Table C3 Index for the extra-curricular use of English 284 Appendices Scale and index documentation: teacher questionnaire Index Frequency of text composition in class or as homework Source Specifically developed for this project Wie oft schreiben die Kinder der 6. Klasse im Englischun‐ terricht oder als Englisch-Hausaufgabe … vt_8.3 … kurze Texte? vt_8.4 … längere Texte (eine halbe A4 Seite oder mehr)? Response categories nie (0), 1-2 Mal pro Jahr (1), 1-2 Mal pro Quartal (2), 1-2 Mal pro Monat (3), 1-2 Mal pro Woche (4), jeden Tag (5) Variable Item values Index characteristics min max n M SD r it α if item deleted vt_8.3 0 4 19 2.74 0.87 0.84 vt_8.4 0 4 19 1.89 0.88 0.84 - Index n = 19 α = 0.91 Table C4 Index for measuring the frequency of text composition in class or as homework 285 Appendix C Index Role of pre-writing activities when teaching writing Source Specifically developed for this project Führen Sie im Unterricht Aktivitäten durch, um die SuS auf das Schreiben eines Textes vorzubereiten? Falls ja, was für Aktivitäten sind dies? vt_21.1 Wir behandeln das Thema, über das die SuS schreiben sollen, im Unterricht. vt_21.2 Wir sammeln gemeinsam Ideen, worüber die SuS schreiben könnten (z. B. mit einem Mindmap). vt_21.3 Wir schauen uns im Unterricht als Beispiel einen ähnlichen Text an. vt_21.4 Wir überlegen uns Wörter oder Sätze, welche die SuS benutzen könnten. vt_21.5 Wir besprechen im Unterricht, wie die SuS einen Text struktur‐ ieren können. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Index characteristics min max n M SD r it α if item deleted vt_21.1 1 4 19 3.42 0.90 0.80 0.90 vt_21.2 1 4 19 2.84 0.90 0.78 0.90 vt_21.3 1 4 19 3.11 0.88 0.75 0.91 vt_21.4 1 4 19 3.16 1.02 0.87 0.88 vt_21.5 1 4 19 2.63 0.90 0.75 0.91 Index n = 19 α = 0.92 Table C5 Index for measuring the role of pre-writing activities when teaching writing 286 Appendices Index Role of instructional scaffolding when teaching writing Source Specifically developed for this project Bieten Sie den Schülerinnen und Schülern während des Schreibens Hilfestellungen an? Falls ja, wie sehen diese Hilfestellungen aus? vt_22.1 Die SuS können einen ähnlichen Text als Vorlage benutzen. vt_22.2 Die SuS dürfen Wörter nachschlagen (z. B. in einem Wörter‐ buch oder im Lehrmittel). vt_22.3 Wenn die SuS beim Schreiben ein Problem haben, können sie mich um Hilfe fragen. vt_22.4 Wenn die SuS beim Schreiben ein Problem haben, können sie andere Kinder um Hilfe fragen. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Index characteristics min max n M SD r it α if item deleted vt_22.1 3 4 19 3.47 0.51 0.14 0.64 vt_22.2 3 4 19 3.89 0.32 0.68 0.37 vt_22.3 3 4 19 3.84 0.38 0.73 0.29 vt_22.4 1 4 19 3.53 0.84 0.26 0.71 Index n = 19 α = 0.56 Table C6 Index for measuring the role of instructional scaffolding when teaching writing 287 Appendix C Index Role of feedback when teaching writing Source Specifically developed for this project Geben Sie den SuS ein Feedback zu dem, was sie ges‐ chrieben haben? Falls ja, wie sieht ein solches Feedback aus? vt_23.1 Die SuS können mir ihre Sätze und Texte zum Durchlesen geben. vt_23.2 Ich gebe den SuS Hinweise (mündlich und / oder schriftlich), was sie noch besser machen könnten. vt_23.3 Ich gebe den SuS Hinweise (mündlich und / oder schriftlich), was sie gut machen. vt_23.4 Während die SuS ihre Sätze und Texte überarbeiten, erhalten sie von mir Hinweise, worauf sie achten sollen. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Index characteristics min max n M SD r it α if item deleted vt_23.1 3 4 19 3.89 0.32 0.59 0.69 vt_23.2 2 4 19 3.68 0.58 0.68 0.58 vt_23.3 2 4 19 3.63 0.60 0.59 0.65 vt_23.4 3 4 19 3.42 0.51 0.37 0.76 Index n = 19 α = 0.74 Table C7 Index for measuring the role of feedback when teaching writing 288 Appendices Singe item Importance assigned to reading Source Specifically developed for this project vt_15 Welchen Stellenwert hat in Ihrem Unterricht das Lesen von längeren englischen Texten (z. B. Texte aus dem Lehrmittel, aber auch Easy Reader u. ä.)? Response categories sehr gering (1) … sehr hoch (7) Variable Item values min max n M SD vt_15 4 7 19 5.58 0.77 Table C8 Single item: importance assigned to reading 289 Appendix C Index Frequency of learning vocabulary Source Specifically developed for this project Wie häufig werden bei Ihnen in der 6. Klasse im Englisch folgende Dinge gemacht: vt_14.1 Im Unterricht neue englische Wörter lernen vt_14.2 Im Unterricht Wörter repetieren, welche die SuS zu einem früheren Zeitpunkt gelernt haben vt_14.3 Spiele, bei denen die SuS englische Wörter kennen müssen vt_14.4 Als Hausaufgabe englische Wörter lernen (mündlich und / oder schriftlich) Response categories nie (0), selten (1-2 Mal pro Jahr) (1), gelegentlich (1-2 Mal pro Monat) (2), oft (1-2 Mal pro Woche) (3), jeden Tag (z. B. im Wochenplan / als Hausaufgabe) (4) Variable Item values Index characteristics min max n M SD r it α if item deleted vt_14.1 2 4 19 2.89 0.66 0.50 0.35 vt_14.2 1 4 19 2.42 0.77 0.59 0.22 vt_14.3 1 3 19 2.11 0.66 0.21 0.57 vt_14.4 2 4 19 2.68 0.75 0.12 0.66 Index n = 19 α = 0.55 Table C9 Index for measuring the frequency of learning vocabulary 290 Appendices Scale Role of orthography in classroom instruction Source Specifically developed for this project Wie gehen Sie im Unterricht mit dem Bereich Rechtschreibung um? vt_19.2 Die SuS müssen die englischen Wörter auch schreiben lernen. vt_19.3 Ich mache Prüfungen, in denen die SuS die englischen Wörter richtig schreiben müssen. Response categories Die Aussage … stimmt nicht (1), stimmt eher nicht (2), stimmt eher (3), stimmt (4) Variable Item values Scale characteristics min max n M SD r it α if item deleted vt_19.2 1 4 19 3.32 0.89 0.77 vt_19.3 2 4 19 3.42 0.77 0.77 - Scale n = 19 α = 0.87 Table C10 Scale for measuring the role of orthography in classroom instruction 291 Appendix C Appendix D Detailed ratings of Profiles 1 - 6 Dimension Rating a Profile number 1 2 3 4 5 6 Final score (RASCH adjusted) b 0.51 1.33 2.15 2.83 3.53 3.82 E-mail overall score c 0.72 1.42 2.23 2.60 3.45 3.80 Coverage 2.00 2.50 3.00 2.50 4.00 4.00 Level of detail 0.00 1.00 2.00 2.50 3.00 4.00 Genre-specific elements of an e-mail 2.00 2.00 3.00 1.00 3.50 4.00 Coherence 1.00 1.00 2.00 3.00 2.50 4.00 Cohesion 0.00 1.00 2.00 3.00 4.00 4.00 Complexity of syntax and grammar 0.50 1.00 2.00 2.50 4.00 4.00 Correctness of syntax and grammar 0.00 1.00 2.00 3.50 4.00 4.00 Vocabulary 1.00 1.00 2.00 2.00 3.50 4.00 Orthography 1.00 2.00 3.00 3.00 3.00 3.00 Punctuation 0.00 2.50 2.00 3.00 3.00 3.00 Story overall score c 0.35 1.20 2.05 3.00 3.45 3.80 Communicative effect / creativity 0.50 1.00 2.50 2.50 3.50 3.00 Level of detail 0.00 1.00 2.00 3.00 4.00 4.00 Coherence 1.00 1.00 1.50 3.50 3.00 4.00 Cohesion 1.00 1.00 2.50 3.50 2.50 4.00 Complexity of syntax and grammar 0.00 1.00 1.50 3.00 4.00 4.00 Correctness of syntax and grammar 0.00 1.00 1.50 3.00 4.00 4.00 Vocabulary 0.00 1.00 2.00 3.00 3.50 4.00 Orthography 1.00 1.00 2.00 2.50 2.50 4.00 Punctuation 0.00 3.00 3.00 3.00 4.00 3.00 292 Appendices Note. a Mean rating of the two raters. Profile 6 was rated by only one rater. b Adjusted to compensate for task difficulty, rater severity and difficulty of the rating criteria using Many-Facet RASCH measurement. c Overall scores calculated by the means of the subcategories Task completion (coverage, level of detail, genre-specific elements of an e-mail OR communicative effect / cre‐ ativity, level of detail), Text structure and cohesion (coherence, cohesion), Syntax and grammar (complexity, correctness), Vocabulary (range) and Language mechanics (orthography, punctuation). Table D1 Detailed ratings of Profiles 1-6 293 Appendix D Multilingualism and Language Teaching herausgegeben von Thorsten Piske (Erlangen), Silke Jansen (Erlangen) und Martha Young-Scholten (Newcastle) Bisher sind erschienen: 1 Jessica Barzen, Hanna Lene Geiger, Silke Jansen (Hrsg.) La Española - Isla de Encuentros / Hispaniola - Island of Encounters 2015, 227 Seiten €[D] 64,- ISBN 978-3-8233-6901-1 2 Anja Steinlen, Thorsten Piske (Hrsg.) Bilinguale Programme in Kindertageseinrichtungen Umsetzungsbeispiele und Forschungsergebnisse 2016, 306 Seiten €[D] 68,- ISBN 978-3-8233-6902-8 3 Christine Möller Young L2 learners‘ narrative discourse Coherence and cohesion 2015, 294 Seiten €[D] 68,- ISBN 978-3-8233-6903-5 4 Thorsten Piske, Anja Steinlen (Hrsg.) Cognition and Second Language Acquisition Research on Bilingual and Regular Language Programs 2022, 386 Seiten €[D] 78,- ISBN 978-3-8233-8194-5 5 Viviane Lohe Die Entwicklung von Language Awareness bei Grundschulkindern durch mehrsprachige digitale Bilderbücher Eine quasi-experimentelle Untersuchung zum Einsatz von MuViT in mehrsprachigen Lernumgebungen 2018, 320 Seiten €[D] 78,- ISBN 978-3-8233-8208-9 6 Cordula Glass Collocations, Creativity and Constructions A Usage-based Study of Collocations in Language Attainment 2019, 306 Seiten €[D] 78,- ISBN 978-3-8233-8171-6 7 Anja Steinlen English in Elementary Schools Research and Implications on Minority and Majority Language Children’s Reading and Writing Skills in Regular and Bilingual Programs 2021, 208 Seiten €[D] 49,- ISBN 978-3-8233-8451-9 8 Patrick Wolf-Farré, Katja F. Cantone, Anastasia Moraitis, Daniel Reimann (Hrsg.) Sprachkontrast und Mehrsprachigkeit Linguistische Grundlagen, didaktische Implikationen und Desiderata 2021, 332 Seiten €[D] 78,- ISBN 978-3-8233-8349-9 9 Ruth Trüb An Empirical Study of EFL Writing at Primary School 2022, 293 Seiten €[D] 58,- ISBN 978-3-8233-8543-1 This book gives an account of the design, implementation and results of the research project “An Empirical Study of EFL Writing at Primary School”, which was carried out in the Canton of Aargau, Switzerland, between 2016 and 2020. The study investigated the writing competence of 322 primary school learners in grade 6 (aged 12-13) in their fourth year of learning English as a foreign language (EFL). Besides gauging the learners’ EFL writing competence and describing the characteristics and qualities of the learners’ texts at different language levels, the study examined current teaching practices and the learners’ perception of EFL writing, and measured the effects of individual and educational factors on the learners’ EFL writing competence. Multilingualism and Language Teaching 9 ISBN 978-3-8233-8543-1